Distributed Artificial Intelligence (DAI)

Distributed Artificial Intelligence or DAI can be defined as a subfield of Artificial Intelligence which, unlike a traditional AI model, follows a decentralized and collaborative approach towards solving problems by breaking a complex problem into smaller sub-problems and handling each one individually.

The models that handle these “subproblems”, in a DAI system, are called the ”agents” or “nodes” which are generally autonomous or semi-autonomous and are located geographically far away from each other. Furthermore, these agents are able to communicate with each other; enabling this collaborative approach and communication requires a specific protocol which is developed by various Decentralized AI networks. The characteristic feature of a DAI agent is its autonomy and ability to communicate and function independently.

Conventional Centralized AI model and Life Cycle

Traditionally, the life cycle of an AI Model is mostly centralized at every stage from data acquisition and learning to deployment. Most Data Science projects developed in the last few decades
follow this standard development pattern and life cycle. Multiple iterations of this end-to-end cycle might be required for creating an efficient model. In most processes involved in this lifecycle, some degree of decentralization is possible.

Disadvantages of Conventional Centralized AI

1) Oligopoly in the AI ecosystem

The Artificial Intelligence industry has been witnessing monopolization in the past few years; a few “technology giants” (primarily Google, Apple, Facebook, Microsoft and Amazon) govern major portions of the industry and are largely responsible for spearheading the AI movement. Numerous upcoming ventures in the field of AI have also been acquired by these corporations. The main reason for this monopolization is the interdependence of various stages of AI development.

Access to huge datasets have enabled industry leaders to develop more accurate and successful AI models and further acquire data for training future models; this has often been termed as the “rich get richer” problem. The ownership of the acquired datasets is centralized; therefore, the large tech companies are the sole owners of these datasets. While these companies are able to successfully train and optimize AI models using these datasets, most new ventures lack the access to such datasets necessary for building a successful model. In a decentralized approach towards AI, multiple agents can contribute to the training of AI models through Federated Learning and Collaborative AI.

It has been argued that this monopolization has been restricting the capabilities of AI in the area of social innovation. Large technology corporations have access to huge datasets, resource capabilities and exceptional professionals in data science and machine learning. Corporations are primarily developing AI for general commercial usage; a decentralized AI network can encourage innovation and efficient usage of AI in resolving major social issues and global challenges since it allows access to large decentrally-owned datasets and most importantly, a medium to collaborate for machine learning and model training.

2) Data privacy issue (centralized learning)

This is one of the most serious problems caused due to traditional centralized AI models and the undemocratic AI ecosystem. As discussed above, major tech corporations have access to large datasets acquired through various sources. Firstly, the ownership of data is centralized i.e. the companies are the owners of the data acquired from users. Secondly, the usage of data is generally not transparent i.e. a user is generally not informed about how and where their data is being used by the corporation. Today, data privacy is a major concern due to various cases of data leaks witnessed in the past few years.

Data privacy issues are particularly harmful when dealing with sensitive data. For example, in order to develop an AI model for hospitals which predicts diseases or deals with particular diagnostics, patient data and medical history would be required. However, medical history is very sensitive information and is vulnerable to misuse. Following a centralized AI approach, the organization creating the model would either fail to secure enough datasets to develop accurate models or would get access to sensitive medical records, both of which are undesirable situations. Increasing cases of data harvesting and data scandals are a matter of grave concern; centralized ownership of data (in traditional AI development) has often been termed as “unethical” and “harmful” as it puts users’ personal data at risk.

3)Unnecessary efforts and reduced efficiency for developers

Due to the absence of frameworks for a decentralized collaborative AI development, traditional practices do not facilitate the sharing of AI models or tools. Due to this, unnecessary reduplication of AI models is required. Most AI tools that predict results and aim to optimize solutions require a collection of numerous models performing various specific functions. Again, large technology corporations have access to various such tools due to huge resources and competent teams of professionals.

Today, almost all industries have some requirement or application for AI services. From healthcare and finance to education and manufacturing, all industries can benefit from the use of AI in understanding consumer behaviours, personalizing features, predicting market responses etc.
However, a significant percentage of businesses are unable to efficiently use AI to benefit their operations because of two reasons. Firstly, pre-existing models that can be purchased are usually not optimized enough for every business to produce desirable results. Secondly, most businesses, excluding large technology corporations, are unable to hire competent teams of AI professionals who are able to create customized AI models to suit their purpose. Through emerging decentralized AI networks, companies could actually combine various semiautonomous AI services to create a custom model, without a lot of effort, which is able to fulfill the requirements.

If the AI ecosystem is democratized, such specific tools could be easily monetized and made available through different frameworks and networks. Furthermore, in a decentralized AI model, the different “agents” or, in this case, AI tools performing specific subtasks are autonomous or semi-autonomous. These agents can communicate with each other and get tasks performed as per the requirement. In such networks, the use of models to get results is monetized i.e. it requires certain “bounty” to be paid to the creator of the model. This problem has been specifically targeted by decentralized AI marketplaces such as SingularityNET. In the past few years, various research teams and developers have been working on creating protocols, frameworks and networks for enabling the interoperability of various AI models to work together and for the developers of these models to be
rewarded for the usage of their services.

Machine Leraning using Decentralized Data

Traditionally, AI Models are trained using centralized data i.e. data stored in a central location. This, of course, raises various privacy concerns since entire datasets are accessible. Recently, various approaches to train AI Models without the need to compile all data in a central location have emerged.

Federated Learning

Federated Learning is one of the most promising techniques aimed at decentralizing the process of AI development. Collaborative or Federated Learning is a Machine Learning (ML) in which AI models are locally trained on various decentralized devices or clients holding their own local data samples. In this approach, data samples are not exchanged or compiled in a central server.With the emergence of IoT (Internet of Things) and the
growth of portable technology, there are billions of devices, which are connected to the internet, around the world. With Federated Learning, the huge data collected by these devices can help train very accurate and efficient AI models without the risk of data leaks and privacy concerns..

Each round of the process of Federated Learning can be summarized in the following five steps:

• The central server selects a statistical model to be trained and a subset of nodes or clients for the training process.
• The central server transmits the same model to all nodes selected (based on specific criterion) in the current round.
• The clients retrieve the model and locally train it using the local user data on the device.
• The nodes send the updated results back to the central server. Results from all selected clients are aggregated to get a final updated model.
• The updates are made to the original model and the process is repeated again with the updated model and a new set of clients.

Notable points related to the process:

• Nodes are usually selected on the basis of numerous factors. For example, in the case of mobile devices battery life, connectivity, network bandwidth, available computation power etc.
• As opposed to various ML training methods, in Federated Learning, datasets and mostly heterogeneous i.e. they differ in magnitude, distribution, etc.
• In a node, the resulting model might be iterated over the local data numerous times before averaging all nodes’ results in the central server. The number of times a client must train the model in each round before sending back the updates depends upon various circumstantial factors. A February 2017 study related to Federated Learning analyzes two algorithmic approaches towards Federated Learning using decentralized data- FedSGD and FedAvg. SGD stands for Stochastic Gradient Descent, a common and effective approach often used in Machine Learning algorithms. FedSGD or FederatedSGD is a federated variant of the SGD algorithm with a few upgrades and additional parameters for batch size, clients etc. FedAvg stands for Federated Averaging algorithm; it works similar to the FedSGD but the nodes perform multiple updates using local data, with weights instead of gradients, before transmitting the results to the central server for weighted averaging.
• Numerous iterations of the above process with different subsets of clients are usually performed in order to achieve desired accuracy. Generally, after a given number of rounds, the model stops showing improvement in accuracy.

The following 4 steps summarize the basic process of model training through this framework:

• Developers upload a lightly pretrained (very limited accuracy) AI model on the Blockchain through a smart contract.
• Contributors test the model by getting predictions for input data without any additional costs.
• Contributors “stake” a required deposit in order to submit a data contribution for training purposes.
• One of the following two scenarios takes place-
• If the data submitted is “good”, the deposit is refunded with some additional incentive.
• If the data submitted is “bad”, the deposit is lost as penalization.

Decentralized Compensation

Since the usefulness or quality of a data provided might be a subjective metric in various cases, such transparency and fairness provided by smart contracts assures the data contributors that they would be fairly compensated for their contributions. The aforementioned compensation is provided through any of the three components of the incentive mechanism-

• Gamification: In this mechanism, a non-monetary compensation is provided to the data contributor. It can be compared to the badges or points awarded on various platforms.
• Reward Mechanism: The contributor is provided some financial rewards based on the improvement in the accuracy of the model due to the data contributed. These financial rewards are supplied from a pool of reward funds
  provided by the development company or any entity that wishes to encourage the training of the model.
• Contributors “stake” a required deposit in order to submit a data contribution for training purposes.
• Deposit, Refund, Penalty: Since a smart contract cannot legally compel a user to pay a penalty, the deposit submitted by the contributors before making the contribution is actually “staked” by them. In case the data provided
  is good, the deposit is returned and financial incentive is provided.