This is a question that has come up many times during AI product discussions – where will the AI live? Does it have to be in the Cloud? Or does it live in the Edge? Is it even an option (sometimes it’s not)? Is there a hybrid solution, that is, bits and pieces live on the Edge, and the Cloud.
There is not much online that compares these today, partly because its still early days. There was a panel discussion on this topic at the Texas Wireless Summit (link to the YouTube video). The excellent panelists covered some of the aspects I talk about here.
The table below summarizes some key considerations and suggests which is better – the Cloud or the Edge. Below we examine each, and present our rationale. There is not a mathematical proof, but rather intuitive arguments, with some examples. We consider two examples. First is the facial recognition on mobile devices – we can unlock our phone, the laptop just by looking at it. The device uses its camera to “see” us, and uses an AI model running on the device itself to recognize the face, and unlock. Why does this model sit on the Edge?
The second example is a drone searching for forest fires. Consider two possible implementations. Option 1 – the images are streamed into the Cloud, where the AI model looks for the fire. Option 2 – the AI sits on the drone itself, and the drone then only communicates the location of the fire.
Accuracy
Accuracy is a simple one. By “accuracy” we mean the accuracy of the AI model’s output. The Cloud has more resources, and in general, that translates into higher accuracy. What are these Resources? – compute and memory, so a bigger, more complex model. But it could also be access to other data (sensor fusion, or data from other deployments), or historical data, or other information that is not available at the Edge.
A drone with a model in the Cloud is likely better at detecting fires.
Time
Time – how long before we get the output result – is slightly more complicated. For the same computation, the Edge will be faster than the Cloud. Why? – the Edge is the data generation point. If the model sits in the Cloud, the data has to be uploaded, then inferred, and the results downloaded back to the Edge. However, if the results are not going back to the Edge and instead stay in the Cloud, then time might be the same.
For the phone unlock example, the Edge is a better place for the AI. The action – unlock the phone – is at the Edge.
Reliability
Sending data to the Cloud has a “variable” time aspect as well. Communication depends on many factors, like network availability, signal strength, data routing, traffic. Some are controllable, others harder. In a worst scenario, what if the data does not get through – will it be an issue that can be recovered from?
As an example, if the face recognition unlock on the mobile phone ran in the Cloud, it likely wouldn’t be a feature!
Power
There is a simple rule – the farther the data has to travel, the more power it needs. The most power-hungry part of an IoT (Internet of Things) device is the radio (eg, see Modeling Power Consumption for IoT devices). The energy consumed of course depends on the amount of data sent. As an example, assume a drone looking for a fire. Option one is to send a video stream and process it in the Cloud. Another would be for the AI model to sit on the drone, and for the drone to only send the location of the fire. From the power perspective, the later will be far better.
Cost
Closely tied to power is the cost. This is not obvious, because cost can include many things. However, purely from a cost-of-data angle, streaming the video is more costly than processing it at the Edge (on the drone, in the example above).
The broader question about cost is harder. One aspect is one-time costs Vs recurring costs. While the Edge device is more a one-time cost, the Cloud presents a recurring cost. That’s only one aspect though, and cost has to be evaluated on a product by product basis.
Security
There are two ways to look at security. One is the security of the data. It is most secure, and most easily secured, if it never leaves the Edge. Once the data is on the internet, or in the Cloud, it is only as secure as the encryption/protocols.
The second is securing the Intellectual Property (IP) – the AI model for instance. This is more secure if its in the Cloud.
Privacy
While “security” looks at the problem from the Provider’s perspective (company building the product), “privacy” looks at it from the Consumer’s perspective (person or entity using the product). It is a major concern, and becoming more important every day. Imagine if every time face recognition AI unlocks my phone, my image is uploaded to the Cloud. Then another AI algorithm uses it to “read” my emotional state, and sends suggestions (ads). That is a possibility (easy) with Cloud-based AI, but not as much with Edge-based AI.
The easiest way to keep our personal data personal is to keep it on our devices, and not send it to the Cloud.
So there’s the short list of considerations to keep in mind while planning your AI deployment, and figuring out where it belongs – the Cloud or the Edge. It is not a comprehensive list, and there are many other considerations like time-to-market, managing deployment, cost of building the solution, technology etc that are not covered. They make more sense though if examined within the scope of the AI project. Feel free to reach out with thoughts or if you need help to get started on your project.