R has gained massive popularity in the past decade as the tool of choice for a wide variety of data analysts. The language is used as part of the data analysis toolchain in some of the biggest companies in the world. The former Revolution Analytics listed up some of the companies that work with R. That list looked impressive already in 2014 including Facebook, Google, Twitter, Monsanto, the FDA, Lloyds, Credit Suisse. Airbnb joined the club in 2016.  

R first appeared in the 1990s in New Zealand, as an implementation of the S statistical programming language. R was written by statisticians, with statistics and data in mind. It is a perfect choice for data analysis, statistical modeling, simulation and graphics. Even though these are key distinguishing features of R, the language provides some other powerful features we will mention below.

Statistics and data in the DNA

R allows you to manipulate (e.g., subset, recode, merge) data quickly.  Some R packages have been designed specifically for these purposes, e.g., dplyr. Typically, a majority of the time spent on an analysis project is spent on the analysis—preparing the data.  R is much adept and efficient in data preparation. Collected data often requires many steps in data processing to be ready for analysis, so R is ideal.

Often, the information that can help answer certain specific questions (churn, cross &-upselling and employee flow) already exist within the company, hiding in plain sight.

It is often a challenge leveraging  data about your  customers or employees for insights to answer questions. If companies could improve their data practices at five important stages, they could become much more effective in solving some of the most pressing problems they face. We think the following steps are important. 

Nebu together with Dutch MOA and British MRS has launched a series of R-scripting training designed for market researchers and data analysts. 

Recently, I was approached by a former business partner, Kees de Jong, a Board Member at NLGroeit, an initiative to provide entrepreneurs peer support. During our meeting, he explained the new initiative and some of the challenges the organization faced.

The scope of the project was clearly going beyond traditional market research: NL Groeit needed to conduct an extensive online assessment, based upon feedback from experienced entrepreneurs, coaches and trainers, and to enrich this data with geolocation based information, such as IP addresses and then provide, automatically, actionable insights for their members to develop and grow themselves, and their organisations. Asked if Nebu could help, I proudly answered "Yes, let's do this. We can deliver the exact data you need."

If I say that in the past 20 years the availability of information has been one of the major game changers, I am sure you will agree. But what information has come available? You will most likely say "about everything". And yes, you are right. Now we all will agree that Google played an important role in that. They made it possible to find it all. I am also sure that Google had no idea on the impact their technology had when they started, nor the users had any idea. But the availability of data only has fueled many businesses over the past 2 decades. You could say you can only start to imagine what you can do with something when you actually can experience it.

You probably heard it before... when working in the Market Research Industry you are frequently confronted with questions like "Can you send me the dataset?". Though this seems to be such a valid and logical question, it also gives you an insight into the typical issues in market research businesses.

This question means that you have to generate a dataset snapshot of a running project. This snapshot is then distributed, and as of that moment, any change in the source of the snapshot (like newly completed interviews, or codings) will not be in the included. This has massive implications on the internal processes of your market research organization, but I talked about it in our previous post.

Waiting for a file or document is something of the past century. Especially in a business process the transfer of information is one of the major obstacles. Ever since Google Docs (and Office 365) have been around we all have learned that collaborate (parallel) access to a document works far better than single (serial) access.

As we learned in our previous blog post, approximately 2.5 trillion million bytes of data are collected - both actively and "passively" - on a typical day here on Planet Earth. (To give you some perspective on the magnitude of that figure, 2.5 trillion million equals 1 quintillion^A or 1 x 10¹⁸.) And as we continue to develop additional data touch points to add to those that already intersect with our lives each day, such as sensors, retail checkout registers, social media, and RFID tags, the amount of data we collect will just continue to expand exponentially.

Collectively, we humans create about 2.5 quintillion (US¹) bytes of data per day; much of this data is created in a "passive" manner, through the myriad data collection and data retrieval sources that we encounter daily. It's not surprising when you think about the growing number of data touch points we have today: Sensors, retail checkout registers, social media, RFID tags .. you name it; it's collecting data. And these devices and sensors continue to proliferate daily.

For those who are employed in a business that "touches" data somehow - whether their company collects data, analyzes it, transmits it, or stores and protects it - "Big Data" is likely a term that has, over the past year or two, risen to the fore of their consciousness. Every component within a company that, for example, manufactures and sells ice cream in its own stores - from the R&D team that develops new flavors, to the marketers who decide which flavors to promote most heavily, to the financial analysts who determine the profitability of each retail location - is very likely a candidate to enter the world of Big Data.

And for those of us in market research, whose livelihoods are immersed an endless cycle of data collection, analysis, transmission, and storage, Big Data has become an inescapable vortex that promises to become more and more integral to the market research process as supplier familiarity increases.