The Impact of the IoT and Data Science on Organizations
The Internet of Things
IOT stands for the “internet of things” and is a recent buzzword heard in the technology landscape as it is a trending topic. Technology has advanced rapidly in the past few years, and microcontrollers are now easily connected to the internet. In other words, linking any object in the real world to the web is easier than ever.
Think of things like a plant that can tweet when it needs watering, a collar for your cat that tracks their location, socks that people with Alzheimer’s wear that alert their family when they’re walking, the Ring.com doorbell, or anything else you can imagine. Gartner projects that by 2020 there will be 21 billion connected devices. (Source, Gartner)
“Makers” are out there building IOT applications of all kinds from inside their garages as well as inside of global corporations. There are even "Maker Faires" popping up everywhere these days. With virtually anything in the physical world being able to send and receive data, organizations are quickly getting loads and loads of data from these connected devices.
A large amount of data is incredible because the useful insight it can provide, which helps to make informed and better business decisions. However, the sheer size of the data poses some challenges for organizations. Where applicable, it’s worth it for organizations to get into the IOT space because organizations can better know, connect with, and serve their customers.
Challenges with Big Data
The most obvious challenge with data coming in from so many devices is how an organization will handle and store the data. One can host the infrastructure on premise, which brings on a particular set of skills needed, or opt to put this information in the cloud, which brings on a different skillset requirement.
There's no right or wrong path to take as there are benefits and drawbacks to each approach. However, being able to store significant amounts of data is just one part of the challenge.
Data Scientists & Machine Learning
With so much data how does one review, analyze, and gain insight from it? The answer to that question is the very reason that the field of data science and the concept of machine learning are currently growing very rapidly.
There’s an increasing demand for individuals with skill sets to work with big data software, to understand how to parse data, learn from it, and make educated decisions. The software can do a lot of this, but often people need to understand these processes and connect all the dots.
Data scientists are skilled at mining and analyzing data in various forms. Data scientists are to the point of being able to accurately predict analytics, behaviors, and statistical trends. Once an organization gets close to predicting outcomes, they’re able to run more efficiently, which helps the bottom line in the end.
There’s also a growing demand for individuals who can code, configure, and understand machine learning algorithms and big data software. Machine learning can be leveraged to aid in this process of making vast amounts of data easier to understand. Recent headlines show that both Google and Microsoft are investing many resources in the areas of machine learning and artificial intelligence.
Whether insights come from humans or machine learning, the ultimate goal is to glean insight from all the data points and make educated business decisions.
Platforms & Tooling Options for Big Data
When it comes to software, tooling, and platforms for handling “big data” there are several options. As imagined, one’s choice of platform may be dictated by their existing in-house technologies and skillsets. Regardless of the technology stack, a full array of solutions are available.
Some solutions offer dynamic scaling to store significant amounts of data while others are more focused on processing the data. In the world of Big Data, a software called “Hadoop” is very popular. Hadoop is an open source programming framework that simplifies processing large sets of data. Many solutions on the market are built on top of or integrate with Apache’s Hadoop software.
Nearly all the major players have developed solutions that are either built on top of, integrate with, or host and connect to Hadoop. Here are a few solutions that work with Hadoop:• Microsoft’s Azure HDInsight
Big Data in Action
One might ask why “Big Data” matters or where can we see examples of big data in use? Big data is being used all around us and as the IOT field continues to grow, we’ll only see it become more relevant.
• Targeting Consumers: Big data is being used to target consumers to sell goods and/or services. A famous example of big data in use was when the retail store Target was able to predict a pregnancy. Based on purchasing trends the store sent coupons for items for a baby before the female even knew she was pregnant. More details on this interesting story.
• Optimizing Business Processes: Big data is also being used to optimize business processes. For example, clothing retailers are now able to more accurately predict the stock they need to keep on hand based on predictions from trends (social media, web searches, and weather). This optimization is possible through big data.
• Fitness: It’s common to see people wearing bracelets that track steps, sleep, heart rate, and more these days. These devices are IOT devices art their core. There are several brands such as the Fitbit, Jawbone’s Up24, and Microsoft’s “Band” to name a few. These bracelets are continuously collecting data on activity and health. These devices typically integrate with one’s phone via an app which stores data in the cloud. Over time one can notice trends in activity (or lack thereof), periods of rest, and calorie burn. When lifestyle adjustments are made based on these insights people become healthier.
• Healthcare: Big data has made a great impact in the field of healthcare. Processes that took days to accomplish in the past now take minutes. Tasks such as decoding DNA strings are examples of functions that have been made much more efficient. This helps doctors to study diseases and develop algorithms to help predict and treat diseases more rapidly.
Big Data & Arrow
Our tool of choice for all big data needs is Microsoft’s Azure. We’ve built several big data applications for our clients. One of the most unique big data projects we’ve done involved large financial data sets and complex tax code. While we can’t dive too deep into the details, we can share that the capabilities that Microsoft’s Azure provides helped with the speed of development, delivery, and results.
We Can Help!
Do you have connected devices overwhelming you with data, need assistance transitioning to the cloud or making sense of your current data? If any of these situations describe you, we can assist!
Contact us to get the conversation started. We’ve helped organizations such as MIT, the Department of Defense, and Hilton Grand Hotels make sense of their volumes of data, and we’d love to help you, too.