Big Data Analytics trends and Sensors’ Role

I attended the Big Data = Big Business Meetup last Thursday and a panel of experts shared their perspectives on the topic “Big Data Solutions – A look into Emerging Tools of the Trade”. It was a good session with 40+ participants.

One of the speakers, Tony Cosentino, VP at Ventana Research, shared the Big Data Analytics trends as follows:

  • Moving from 20th century designed data to 21st century organic data; from confirmatory analytics to exploratory analytics
  • Moving from sample type of analytics to sensors type. Analytics and data are coming together into one environment instead of being separate.
  • Moving the conversation from data to outcome or business orientation

I was particularly interested in Tony’s speech so I did some research about these trends as follows:

  • Designed data vs. organic data:

This Census Bureau’s blog explains that the Census Bureau has created “designed data” based on pre-specified purpose. In contrast, data collected  through internets, sensors and other systems are organic data. The blogger believed “The combination of designed data with organic data is the ticket to the future”.

  • Sample data vs. sensor data:

Sample analytics is used widely in the conventional market research. The research population is generally too big to be covered in a survey; therefore, researchers usually choose a portion of the population (i.e. sample) to do a survey. The sample size and selection are carefully determined in order to capture the representation of the whole population.

A sensor is a device that measures a physical quantity / activity and transforms it into a digital signal. Sensors are always on, capturing data at real time and powering the “Internet of Things.” Sensors can collect enormous data and Cloud computing and storage help to make the analytics possible.

  • Conversations on data vs business:

Data itself is not the focus of the conversation anymore. Nowadays, the business value provided by the big data is the focus.

I agree that the combination of organic data and design data will create valuable data. I believe we need to have a sampling mechanism with organic data since the volume is big. For example, NFC is one of the sensors. When the technology takes off, it will provide interesting data sets. How to translate the data into value added information for businesses take specific design.

This Hadoop blog suggests that “sensors can be used to collect data from many sources, such as:

  • To monitor machines or infrastructure such as ventilation equipment, bridges, energy meters or airplane engines. This data can be  used for predictive analytics, to repair or replace these items before  they fail.
  • To monitor natural phenomena such as meteorological patterns, underground pressure during oil extraction or patient vital statistics during recovery from a medical procedure.”

I think sensors go beyond these domains. For example: an NFC embedded wearable device can monitor body movements and vitals, such as heart rate and blood sugar. Digital health and fitness mentioned in a blog of Aaron Rose  is possible because of the sensors. The Fujitsu NFC smart glove shows a use case beyond digital health and there is unlimited space for monitoring these types of innovations.

These thoughts were triggered by a two hour Meetup. Can you imagine what thoughts will be triggered in two days? I am looking forward to attending the Big Data Innovation Summit held in Santa Clara on April 9 and 10th. With 80+ sessions, it will definitely broaden my vision and expand my imagination.

What are your thoughts on these trends?

Big Data Innovation Summit 2014

Big Data Innovation Summit 2014

Take Control of Your Data

“Big Data” and “analytics” have been in a lot of conversations lately. Many businesses want to jump on the wagon to use big data for marketing and product development. At the same time, most of the companies don’t really know what to do with a flood of data. How to capture, analyze and utilize this data for business insights is critical.

O’Reilly published an ebook titled Business Models for Data Economy, and it is a good reference book for this topic. It says, “Whether you call it Big Data, data science, or simply analytics, modern businesses see data as a gold mine.” Indeed, we live in a fast-evolving world of data economy. Forrester predicted that 2014 would be the year that marketing leaders will put insights to use.

My interest in Big Data started while I was writing my bookEveryday NFC”. I realized that sensors play a crucial role in helping big data move forward, and I wanted to learn more about Big Data. Going to a conference on the topic was a quick way for me to pick up the terminologies, use cases and visions. Networking with early adopters and watching their progress were good learning points. After this initial overwhelming learning experience, I was able to dive deeper into a specific area that applied to my work.

Here is an opportunity for you to learn about Big Data or to share your experience.

The Big Data Innovation Summit will be held in Santa Clara on April 9-10 in 2014, to guide you in taking control of your data. The Summit brings leading thinkers together for presentations, workshops and panels. To network with these people would be a big win in the Summit. There are four tracks: Data Analytics, Hadoop & No SQL, Data Science and the Machine Learning & Algorithms. You can view the schedule here. Follow #datawest14 and @IE_BigData for further updates.

Are you ready to jump on the wagon?


Reflections on Big Data Confereneces

Last week, I attended two technical events. One was Big Data Techcon in San Francisco and another was Seattle Biz-Tech Summit 2013. My focus was on Big Data and proximity sensors.

For Big Data Techcon, there were many sessions about tools; for example, how to collect data, analyze the data and make a correct interpretation of the analytics. The emphasis is on engineering data. Two things that stood out for me was the graph data base and the keynote speech by Doug Cutting.

The graph data base has an advantage to visualize the connections between Big Data. The book, “Graph Databases”, was given away at the session led by Max De Marzi. He was passionate about Neo4j and showed us the connections between Facebook accounts using code. The connections were visualized regardless of the privacy setting in Facebook.

In the keynote speech, Doug Cutting, the founder of Hadoop claimed that “Hadoop 2 is the Big Data OS” and “Open source’s time has come”. After the keynote, Doug was available to talk to people who wanted to obtain his insights or wanted to have a photo with him. Regarding my inquiry about his view on proximity sensors and Big Data, he saw the significance of the sensor impact to Big Data and made an example with retail stores “What would be the value to the retail stores when they can figure out the shopper’s favorable route.

Seattle Biz-Tech Summit 2013 also focused on Cloud and Big Data. I particularly enjoyed the panel “Innovation and Impacts of Cloud Computing and Big Data”. Dave Segleau, Director, Oracle described the phases of the customer adoption of Big Data as:

  1. What is Big Data?
  2. What can Big Data do?
  3. I have a Big Data (or NoSQL) problem. How can I use your product help me build and deploy a Big Data (or NoSQL) based solution?
  4. I’m starting to understand the issues (limitations, requirements, administration) around managing a Big Data (or NoSQL) solution.
  5. Here’s how I can leverage Big Data to benefit the Enterprise and our customers.

Ying Li, Director, ACM SIGKDD suggested that we would move from an engineering data phase to a data knowledge sharing phase in the future. She was an advocate for open data. Jay Mozek, Chief Architect & Director, iSoftStone thought that we need to be clear about the business goal before engineering data. Chris Garvery, Senior Director, Expedia encouraged us to think what we can’t do today and use data to discover the possibility. Panelists had their own perspectives and their unique views made the session informative and interesting.

Yesterday, I found this article that shared how graphics chips can help process big data sets in milliseconds and “opening up new ways to visually explore everything from Twitter posts to political donations.” This trend of facilitating big data visualization is certainly in full swing.