close

Artificial Intelligence

Architecture PatternsArticlesArtificial IntelligenceDeep LearningIOT

Applying deep learning and computer vision on the Edge Gateway

Edge gateway is about providing real-time decision making at the edge. In this article, I will talk about how to apply computer vision at the edge for various security and surveillance activities. For instance, you would like to detect suspicious activities at the ATM, simple cases like people wearing helmets and entering the ATM to unusual movements or would like to capture and identify objects in remote areas using Drones like crops length to mining activities. In such scenario, you would need real-time decision making at the edge as it might not make sense to transfer the data over the cloud for processing, be it for latency issues, bandwidth, cost issues or there might be none or very limited connectivity. In this case you employ intelligence at the edge of the devices.

edgegateway-useIn order to build out the solution, you need to employ computer vision algorithms on the edge. You can build this using commercial available API’s or using various open source deep learning framework like Theano, TensorFlow, Café etc.  Deep learning is a branch of machine learning for learning multiple levels of representation through neural networks.  Neural network in image processing, automatically infer rules for recognizing images instead of you extracting thousands of features to detect images.

Various deep learning architectures are available like convolutional neural network, recurring neural network to solve specific problems like computer vision, natural language processing, speech recognition and achieves state of the art results.

For Computer Vision, we specially use a CNN network to identify images or use of the pre-trained instance (like Inception model released by Google which is a 22 layers’ deep network which achieves start of the art results for classification and detection for images). For a computer, images are nothing but a vector or set of numbers/pixels. Convolutional Neural Networks learns features automatically using small frames of equal size images and each network gets the same input and interesting pieces are extracted (for more details, please refer to how CNN works).  The extract of interesting pieces (which are again vector of numbers) is the heart of how CNN works. For instance, in case of helmet, some network would learn a round edge, some may learn a glass pane in the front and so on. The idea is irrespective of where the object is in the frame, you should be able to identify to image.

Prior to CNN for image detection, you would need to crop the images and provide area of interest. For instance, if you are detecting various categories of birds, you would usually crop the bird image and remove any surrounding context, like trees, bushes, sea, sky etc and provide the bird image only. With CNN, the idea is to train with those images and let network figure this out (though cropping may help in some cases for increasing accuracy). Based on my experiments, the CNN is able to predict object most of the times which have surrounding context, but with lower accuracy. As mentioned, the idea is to identify objects irrespective of where they are found in the image and I am sure lot of research is going on to improve the CNN networks. Having the right training data (images and label) is a must for training networks with such variations.

edgeo

 

On the left hand side, is the stack view of employing deep learning API on the edge gateway. It consists of API, the learning model and the classification service which is used for classification of objects. There are lot of innovations happening to optimize the use of deep learning libraries on the edge.

 

 

 

So how do you go about implementing this. I will talk about one approach of building this out –

  1. Build your own CNN or Start with a pertained CNN network (like an Inception Model)
  2. Get the training and test data (images and labels)
  3. Train or retrain (i.e transfer learning) the network.
  4. Optimize with learning rates, batch size etc. for desired accuracy. Get the trained model
  5. Use the trained model for classification.
  6. Deploy TensorFlow API and trained model on the edge  (you can package this in docker)
  7. Use the classification code on the edge gateway to detect objects.

The following is part of my presentation that I delivered at IoTNext. I will update the article with the youtube video once available.

read more
ArticlesBooksCognitive Computing

Building Cognitive Solutions – A Definitive Handbook

Let me first start with an introduction on “Cognitive Computing”. Cognitive computing are systems that are designed to make computer’s think and learn like human brain.Similar to an evolution of a human mind from new born to child to an adult, where new information is learned and existing information is augmented, cognitive system’s learn through the vast amount of information fed to it and training on a set of information,so it can understand the context and help in making informed decisions.

For example, if you look at any learning methodology, a human mind learns and understand the context, but its equipped to answer questions in an examination/interview which it might not have seen before, but using the experiences and past learning,a informed judgement can be made. Similarly cognitive systems are modelled to learn from past set of reference data set (or learning) to help users make informed decisions. Cognitive systems can be thought of as a non programming system which learns through the set of information, training, interactions and a reference data set.

From a technology perspective, at a very high level, building a cognitive systems requires technologies that can understand the language,context, entities and relationship (NLP), learn through a set of supervised or unsupervised learning (Machine learning methodology), domain adoption through various techniques, technologies to help source, curate and manage content, runtimes to  build out the components together in a loosely coupled manner and wide variety of  tooling’s and methodology to enable making cognitive applications. I envision most of the cognitive capabilities offered as service over a cloud (a marketplace for cognitive and smart apps) which can be used individually or as a composite to create applications. The key here is domain adaptability, else we are looking at a general purpose AI system, which in my view would not provide precise and accurate suggestions or predictions. Most of the first generation cognitive services are focused on providing API without any provision to train or adapt to use cases. Even if they provide a provision for training and recognizing new terms etc, based on my experiences it doesn’t work out well. For instance, take the example of Google NLP, Watson NLP APIs or any open source NLP framework like standford or Apache NLP, which provides general NLP parsing (based on wikipedia it can recognize common terms etc..), but fails to solve any real use case on its own. The point is you can’t just rely on the bare APIs, you need to build upon it to solve for any real use case. When it comes to solving enterprise solutions,you are looking at precise suggestions at the top and most of AI engines in the market are actually general purpose AI , which fails to get the level of accuracy required from an AI system. Even if you train a general purpose AI, there are “n” number of factors to get the required level of accuracy. I haven’t seen a AI system or design, which is build from the ground up to make it easy for end consumers, enterprises or users to adapt to required use cases.

Through my upcoming book – “Building Cognitive Solutions – A Definitive Handbook”, I would share my experiences on building a cognitive solution, the right way. There are lot of misconceptions on how to build cognitive applications and this would be the first practical guide on building cognitive solutions.

I plan to show a general methodology to build cognitive applications and the recipe to build and end to end cognitive solution. The book would also cover “deep learning” and new approaches to build cognitive solutions. This would follow the same style as my earlier handbook -” Enterprise IoT – A Definitive handbook

I am looking out for contributors/co-author for my book, who are experts on deep learning and would like to contribute and share their knowledge with the wider audience. Kindly reach me at me@naveenbalani.com for more details.

read more
ArticlesMachine Learning

Introduction to Machine Learning

Following is the Wikipedia definition of Machine Learning –

“Machine learning explores the study and construction of algorithms that can learn from and make predictions on data.”

In simple terms, machine learning is how we make computers learn from data using various algorithms without explicitly programming it so that it can provide the required outcome – like classifying an email as spam or not spam or predicting a real estate price based on historical values and other environmental factors.

Machine learning types are typically classified into three broad categories

  • Supervised learning – In this methodology we provide labeled data (input and desired output) and train the system to learn from it and predict outcomes. A classic example of supervised learning is your Facebook application automatically recognizing your friend’s photo based on your earlier tags or your email application recognizing spam automatically.
  • Unsupervised learning – In this methodology, we don’t provide labeled data and leave it to algorithms to find hidden structure in unlabeled For instance, clustering similar news in one bucket or market segmentation of users are examples of unsupervised learning.
  • Reinforcement learning – Reinforcement learning is about systems learning by interacting with the environment rather than being taught. For instance, a computer playing chess knows what it means to win or lose, but how to move forward in the game to win is learned over a period of time through interactions with the user.

Machine learning process typically consists of 4 phases as shown in the figure below – understanding the problem definition and the expected business outcome, data cleansing, and analysis, model creation, training and evaluation. This is an iterative process where models are continuously refined to improve its accuracy.

machine-learning-process

From an IoT perspective, machine learning models are developed based on different industry vertical use cases. Some can be common across the stack like anomaly detection and some use case specific, like condition based maintenance and predictive maintenance for manufacturing related use cases. For more details on use of machine learning in context of IoT, refer to my book – http://amzn.to/2cHuAt1

read more
ArticlesBooksCognitive ComputingFeatured

Cognitive IoT Book

Cognitive Internet of Things is about enabling current IoT technologies with human-like intelligence. Cognitive systems in the context of IoT would play a key role in future.

Download this short ebook (under 15 pages)  to get an overview on Internet of Things and Cognitive Computing, its architecture and synergies between them.

cognitive-iot-book

The book comprises of 4 parts –

1. Introduction to Internet of Things.
2. Introduction to Cognitive Computing.
3. Cognitive Internet of Things
4. Architecture of a connected world.

Download free or buy from –

1. Smashwords –  Navigate to this link – https://www.smashwords.com/books/view/598152 and apply coupon code – SS49Y to get it for free.

2. Amazon Kindle  Store-  Navigate to this link – http://amzn.to/1XOcSd2 and purchase it. (Amazon doesn’t allow free listing 🙁  )

Enjoy reading it.

read more
ArticlesCognitive ComputingIOT

Cognitive IoT Ebook Free Download

“Internet of Things is a vision where every object in the world has the potential to connect to the Internet and provide their data so as to derive actionable insights on its own or through other connected objects“

“Cognitive computing are systems that are designed to make computers think and learn like a human brain. Similar to an evolution of a human mind from a newborn to teenager to an adult, where new information is learned and existing information augmented, cognitive system learn through the vast amount of information fed to it. Such a system is trained on a set of information or data so that it can understand the context and help in making informed decisions.”

Cognitive Internet of Things is about enabling current IoT technologies with human-like intelligence.

This is a short ebook under 15 pages targeted to get an overview on Internet of Things and Cognitive Computing and synergies between them.

The book has 4 sections –
1.Introduction to Internet of Things.
2.Introduction to Cognitive Computing.
3.Cognitive Internet of Things
4.The architecture of a connected world.

Click Here to download this ebook free.

Enjoy reading it.

read more
ArticlesCognitive ComputingCognitive RetailFeatured

Cognitive Retail Platform


Cognitive Retail Platform

Happy to announce Cittin Cognitive Retail Cloud Platform , an end-to-end cognitive, digital marketing and analytical platform for retail.

Checkout Peffy – The personalized cognitive shopping application powered by Cittin Cognitive Retail Cloud Platform, which showcases some of our platform cognitive services.


Checkout the youtube video for more details on the Peffy Application

From a technology perspective, the Peffy application showcases the use of various machine learning algorithms, cognitive computing, natural language processing, personalisation  and visual search through the usage of Cittin Cognitive Retail platform. Checkout the live demo at http://peffy.in/
Stay tuned for more updates.

For queries and questions, reach out to me at naveen@cittin.io

read more
ArticlesCognitive ComputingIOT

Cognitive Computing, IoT and Synergies

Let me first start with an introduction on “Cognitive Computing”. Cognitive computing are systems that are designed to make computer’s think and learn like human brain.Similar to an evolution of a human mind from new born to child to an adult, where new information is learned and existing information augmented, cognitive system’s learn through the vast amount of information fed to it and training on a set of information,so it can understand the context and help in making informed decisions.

For example, if you look at any learning methodology, a human mind learns and understand the context, but its equipped to answer questions in an examination/interview which it might not have seen before, but using the experiences and past learning,a informed judgement can be made. Similarly cognitive systems are modelled to learn from past set of reference data set (or learning) to help users make informed decisions. Cognitive systems can be thought of non programming systems which learns through the set of information, training, interactions and a reference data set.

From a technology perspective, at a very high level, building a cognitive systems requires technologies that can understand the language,context, entities and relationship (NLP), learn through a set of supervised or unsupervised learning (Machine learning methodology), domain adoption through various techniques, technologies to help source, curate and manage content, runtimes to  build out the components together in a loosely coupled manner and wide variety of  tooling’s and methodology to enable making cognitive applications. I envision most of the cognitive capabilities offered as service over a cloud (a marketplace for cognitive and smart apps) which can be used individually or as a composite to create applications. Best way to get started with building cognitive applications, its through the Cognitive Services Zone on bluemix

So where does Internet of Things fits into the cognitive world. I assume you are aware of IoT, if not, please read my introductory blog at – IOT – Are you connected?

Let’s start with high level view of an IoT Cognitive Systems Architecture. The real value of IOT applications is realized by enabling them as part of existing applications and higher value added services to create new innovative business solutions.

IOT-COGNITIVE-COMPUTING-ARCHITECTURE

IoT is not just limited to devices and sensors, but how do you derive real insights from the sensor data, filter and analyze the data and use the information with multiple data sources, understand and infer relationship in context of the use case to augment current application with new offerings or build new innovate solutions. The architecture talks about the evolution of an application, building on the value chain and how various cloud services and offering are used to enable smarter applications.

Let’s take an example of a cognitive IoT application. I have taken a very simple example to get the technology and benefits across.

1. I step out of my home and the home electricity turns into a power saver mode.
2. I step into the car and the car recognizes me.
3. My car seats are automatically adjusted.
4. My favorite music station is set and play list is started.
5. An aggregated news for the day is available and tailored for me on my dashboard and read it out by the dashboard device. If I am travelling, weather forecast and news related to the place I am travelling is available as an add-on.
6. As I pass by malls, my car reminds to purchase stuffs.
7. I sync my digital cart and get my purchases quickly. No need to move around, find things and put it to a physical cart.
8. I come back to my car, I start interacting with the Car in Natural human language instead of typing in numbers and searching for things.
9. I ask for good places for lunch which I haven’t visited. Based on my past experience’s and cuisine preference,ratings from third party sites, a set of recommendations are provided. I choose one of them.
10. GPS devices are synced up based on my response and direction’s changed.
11. Alternative route is automatically selected based on traffic sensor’s which the GPS device is subscribed to.
12. I have lunch, I don’t need to carry cards, I am being recognized. The best credit card (after checking available offers for that restaurant) from my digital wallet is automatically selected and used for the payment. Welcome to smart cashless transactions.
13. I come back home, lights are back up again, all devices started.
14. I say “good night”, lights are dimmed and tell my clock to wake me up at 7:00 am after checking my flight status.

As you see in the above example, the real value is derived from how data from sensors are used as part of the broader ecosystem and how cognitive capabilities and learning are used to provide value added services.

These are just one of the many experiences. The real experiences would be technology touching our every day lives and making it easier and a better place to live. I gave a consumer driven example, but IoT benefits can be leveraged across industries – connected cars, aviation, manufacturing, predictive maintenance, healthcare, insurance etc..

With the context set, I suggest to read my follow on blog on “Architecture of a Connected World“.

read more
Architecture PatternsArticlesCloud ComputingCognitive ComputingIOT

IOT Cognitive Systems Architecture

Following is a high level view on how I visualize an IOT Cognitive Systems Architecture. The real value of IOT applications is realized by enabling them as part of existing applications and higher value added services to create new innovative business solutions.

Over the next few years, most of the capabilities (leaving out some of the hybrid cloud use cases) would be available as cloud services, enabling creating of smart business applications through a plug and play architecture. I evasion an ecosystem of cloud services (much like an apple or an android app/play store), where applications can be assembled from multiple providers to provide higher value solutions.

IOT-COGNITIVE-COMPUTING-ARCHITECTURE

PS- The above are some of my rough drafts on my upcoming book on “Advanced Computing – A Futuristic View” which covers IOT, Cognitive, Big Data, Cloud Services, Analytic and Smart Applications.

read more
ArticlesCloud ComputingCognitive ComputingIOTUpcoming technology trends and reviews

Smart Apps, IOT and Cognitive Computing

Smart Computing – IOT meets Cognitive Computing

A peep in the future of connected world –

1. I step out of my home and the home electricity turns into a power saver mode.
2. I step into the car and the car recognizes me.
3. My car seats are automatically adjusted.
4. My favorite music station is set and play list is started.
5. An aggregated news for the day is available and tailored for me on my dashboard and read it out by the dashboard device. If I am travelling, weather forecast and news related to the place I am travelling is available as an add-on.
6. As I pass by malls, my car reminds to purchase stuffs.
7. I sync my digital cart and get my purchases quickly. No need to move around, find things and put it to a physical cart.
8. I come back to my car, I start interacting with the Car in Natural human language instead of typing in numbers and searching for things.
9. I ask for good places for lunch which I haven’t visited. Based on my past experience’s and cuisine preference,ratings from third party sites, a set of recommendations are provided. I choose one of them.
10. GPS devices are synced up based on my response and direction’s changed.
11. Alternative route is automatically selected based on traffic sensor’s which the GPS device is subscribed to.
12. I have lunch, I don’t need to carry cards, I am being recognized. The best credit card (after checking available offers for that restaurant) from my digital wallet is automatically selected and used for the payment. Welcome to smart cashless transactions.
13. I come back home, lights are back up again, all devices started.
14. I say “good night”, lights are dimmed and tell my clock to wake me up at 7:00 am after checking my flight status.

These are just one of the many experiences. The real experiences would be technology touching our every day lives and making it easier and a better place to live.

read more
ArticlesCognitive Computing

Invoking Alchemy Face Detection service using JQUERY

Here is a sample reference on how to invoke Alchemy Face Detection service using JQUERY.  The details were not available on the website, so I decided to build one quickly. I would use this only for quick experimentation as it exposes the client key on the browser side.

Here is the index.html file –

<!doctype html>
<html lang=”en”>
<head>
<meta charset=”utf-8″>
<meta http-equiv=”X-UA-Compatible” content=”IE=edge,chrome=1″>
<script src=”http://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js”></script>
<script src=”script.js”></script>
</head>
<body>

<h1>File Upload</h1>

<form action=”#” method=”post”>
<ul>
<li>
<label for=”name”>Name:</label>
<input type=”text” name=”name” id=”name” multiple>
</li>

<li>
<label for=”file_upload”>File:</label>
<input type=”file” name=”file_upload” id=”file_upload” multiple>
</li>

<li><input class=”button green” type=”submit” name=”submit” value=”Submit Content”></li>
</ul>
</form>
</body>
</html>

Here is the Javsacript file (script.js) code

$(function()
{

var files;

// Add events
$(‘input[type=file]’).on(‘change’, prepareUpload);
$(‘form’).on(‘submit’, uploadFiles);
function prepareUpload(event)
{
files = event.target.files;
}
function uploadFiles(event)
{
event.stopPropagation();
event.preventDefault();

$.ajax({

url :’http://access.alchemyapi.com/calls/image/ImageGetRankedImageFaceTags?apikey=yourkey&outputMode=xml&imagePostMode=raw’,
type: ‘POST’,
data : files[0],
cache: false,
processData: false,
contentType: ‘application/x-www-form-urlencoded’,
success: function(data, textStatus, jqXHR)
{
if(typeof data.error === ‘undefined’)
{
console.log(‘Data: ‘ + jqXHR.responseText);

}
else
{
console.log(‘ERRORS: ‘ + data.error);
}
},
error: function(jqXHR, textStatus, errorThrown)
{
console.log(‘ERRORS: ‘ + textStatus);

}
});
}

});

read more
1 2 3 4 5
Page 4 of 5