What new data collection methods are being used for market research?

Maintaining ethical standards and safeguarding privacy are paramount in the context of passive data collection. It is imperative that consumers or clients are fully apprised of the accessibility of this information to ensure transparency. Awareness, often undervalued in research, plays a pivotal role in this domain. An abundance of publicly available information holds the potential to evolve into valuable data. This approach is perceived as a positive initiative by consumers and clients, wherein the collection of public, legal, and openly accessible data underscores a commitment and vested interest in their well-being.

Collecting data (passive data collection) without the user's awareness as they navigate through a website is a common practice accomplished through the utilization of cookies, web beacons, or other identification methods. The advantages of passive data are: 1. By relying on passive data, we eliminate the potential errors that may arise from individuals reporting their own behavior, ensuring the accuracy of the final data. 2. Not only does this approach alleviate the burden on respondents, but it also eliminates the need for them to answer inquiries about their digital behavior.

Ethics & Transparency when collecting and using passive data is the biggest challenge by far. At the moment is still a Wild Wild West situation where companies are able to hide behind complex T&C and collect way much more data than its appropriate or even needed. Beyond ethics and transparency: Data quality matters: Assess potential biases, incomplete datasets and limitations of passive methods. Clean data = clear insights Context is key: Combine passive data with surveys to understand motivations behind observed behaviours Respecting preferences: Offer opt-in/out options to give users a choice Explain the why: Communicate research's goals & benefits of passive data, educate consumers about how their data informs better products & services

Passive data collection offers accuracy by recording user behavior, reducing self-reporting bias. It can collect large volumes of data quickly & continuously, without needing active consumer participation. Privacy concerns are there - data collection without explicit user consent can lead to privacy issues & potential non-compliance with data protection regulations. Data overload can take place, making data handling & analysis resource-intensive. Lastly, passive data can sometimes be misinterpreted without proper context. For eg, high website traffic might not equate to high engagement. Therefore, while passive data collection can provide valuable insights, businesses must balance this with ethical considerations & consumer trust.

Passive Data Collection: In market research, passive data collection means gathering info without people actively participating. Overview: Passive data collection involves observing user behavior without them doing anything special. How to do it: Use tools like sensors, smart devices, or software to track actions and gather insights. Benefits: Get real-life, authentic data without relying on self-reported information. Challenges: Privacy concerns can arise, so it's crucial to follow ethical guidelines. Examples: Tracking website clicks, monitoring app usage, or studying physical movements with sensors.

Mobile Data Collection: Using mobile devices like smartphones or tablets to gather information for market research. Overview: Leverage the widespread use of mobile devices to collect data conveniently. How to do it: Develop mobile-friendly surveys, apps, or use location-based services. Benefits: Reach a large audience, get quick responses, and access real-time data. Challenges: Ensure surveys are concise for mobile users, considering smaller screens. Examples: Conducting surveys through mobile apps, tracking GPS data for location-based insights.

MDC is a powerful way to gather information, and as researchers, we're still exploring its full potential. The way clients and consumers use mobile tech offers tons of great insights for us. But, we've got to remember to tweak our approach too. For example, creating a simple, easy-to-use system for collecting data on the fly.

Mobile data collection used to be very costly but in the last several years the costs to collect data has declined quite significantly. Using SMS, apps, social and other mobile-focused data collections has allowed the research industry to make massive strides in securing research faster - especially among younger groups of survey participants. Using the correct tools and following legal privacy matters is now the only real 'hurdles' but these components are now being fixed and adhered to. This allows mobile data collection to increase response rate and complete surveys faster. A win for all of us.

Mobile data collection is transforming the market research niche of mystery shopping, enabling brands to receive real-time feedback on key customer touchpoints - in-store/on-site, online, or phone interactions. Mobile market research mystery shopping supports companies in understanding how they are performing, according to their brand standards, at scale, and with granular detail. Quantitative performance metrics are easily enhanced with audio, video and image capture, providing a comprehensive understanding of customer experience, sentiment and satisfaction. Leading research providers arm their field researchers with secure mobile data collection apps, with surveys and reports highly customized to the unique needs of each client.

Pew estimates that now-a-days more than 60% of the quant surveys are taken on mobile. So mobile is no longer the alternative its the main platform

Social media is a new tool for market research and sentiment analysis is especially useful. It tells us how people feel about different products or brands. But we have to be careful. Some social media posts are made by bots, not real people, which can make the data less trustworthy in several cases. We need to check that we're analyzing real customer opinions.

Imagine a company that sells sports equipment. They decide to use social media data collection as part of their market research strategy. They start by analyzing posts and comments on various platforms. They look for keywords, also engage users in conversations and run contests to increase interaction and gather more data. Through this process, they notice a growing trend of people talking about eco-friendly sports equipment. Many users express a willingness to pay a premium for products made from sustainable materials. The company then decides to launch a new line of eco-friendly sports equipment, which turns out to be a great success.

Social listening is the hot measure. The marketers dream is something going viral so measuring the impact on social media continues to become more of a priority. The big benefit is that posts etc are freely available and the cost is tools to make aggregation easier. Alongside this the data source is constantly being updated. The biggest watch out here is fake accounts. I can't think of any social media that is not plagued by bot accounts and they make fake posts to create an air of legitimacy.

Social media data collection can be gathered through numerous social media platforms. The definition itself refers to social media demographics, user intent and much more - collected through analytics tools on social media. It also is data collected from content individuals post publicly on social media. Social media listening tools secure this type of data – usually assisting with marketing and sales. This type of collection method has very similar drawbacks to the early days of online and telephone methods. I anticipate this will become much more of a tool for researchers once the bias, geography, etc. issues get resolved. I doubt it will take long as this is a new an important focus for the industry.

Beyond noise & bias that the AI-written article talks about, here are few other points to keep in mind: Ethical considerations: Tread carefully. Respect user privacy, avoid any manipulation and be transparent about data usage Fake news & bots: Fact-check, filter, and employ AI tools to spot misinformation and automated accounts Shifting trends and dynamics: Stay alert to platform changes, emerging communities, and evolving slang – your methods need to adapt Qualitative nuances: Sentiment analysis is great, but combine it with human analysis to capture sarcasm, context, and cultural differences Focus on value extraction: Don't just collect data, analyse it creatively to reveal hidden patterns, user journeys and unexpected insights

One of the significant advantages of video data is its ability to capture nonverbal cues. It can record the interaction of the participant with the environment, providing a permanent record.

Here are some things many people don't know about video data collection... With the right technology partner, you can detect people's emotions from their speaking voice (not their facial features or the words they use). There are at least 4 advantages to that approach over traditional video tech. You can: 1) Capture it via a low-cost add-on to your preferred survey platform, upload file and/or see emotions live in real time, 2) Give people the option to answer via audio when there isn't a good environment for video and get the same results, 3) Get automated transcriptions and translations, and 4) Gain all the additional insights without forcing people to stay in front of the camera so their eyes can be tracked, expressions seen, etc.

Video data collection to me means recording an interview. It’s not a total answer to a study or survey but a component of it. Body language, expressions and more can be viewed and provide data - but it should remain in a supportive role and not be featured. It does help, but don’t make video your primary source. The questions and interaction should be the primary source.

Video Data Collection: Using videos to collect data, such as consumer reactions, preferences, or usage patterns. Overview: Capture visual information to understand consumer behavior. How to do it: Conduct video interviews, analyze user-generated videos, or use eye-tracking technology. Benefits: Gain a deeper understanding of non-verbal cues and reactions. Challenges: Ensuring consent for video recording and handling large video datasets. Examples: Recording focus group discussions, analyzing facial expressions in product testing.

A consumer goods company conducts video diaries where participants record themselves using a new product over a week. Through video testimonials, the company gains insights into the actual usage, challenges, and satisfaction levels of their product, enriching their understanding beyond written feedback.

Time-saving is a significant advantage in AI data collection, yet it comes with potential downsides like errors and a loss of control over insights. Every research project requires vigilant oversight. When embracing new tech like AI, meticulous understanding and control over these systems are crucial. A lack of comprehension could result in unnecessary data or errors, causing researchers to squander considerable time—the AI paradox.

This approach will be a major part of the future of market research. It is currently in the very early stages. I’m not yet comfortable with it just yet, but no doubt it will significantly improve and at a rapid rate. I think by 2025 I’ll be eating my words re: usage. At the moment I believe people don’t understand the legal components of AI & research participation. The future is bright for using AI but we aren’t there yet.

The use of such technologies further emphasizes the need for robust data privacy measures. Businesses must ensure they are using these technologies in a way that respects consumer privacy and complies with data protection regulations. This includes anonymizing data and implementing stringent data security measures.

AI Data Collection: Utilizing artificial intelligence to analyze large datasets and draw meaningful conclusions. Overview: Let AI algorithms process vast amounts of data to uncover patterns. How to do it: Implement machine learning models for predictive analysis or automated insights. Benefits: Increase efficiency, accuracy, and discover hidden trends in data. Challenges: Ensuring data quality and avoiding biases in AI algorithms. Examples: Predictive analytics for sales forecasting, using AI to analyze customer reviews.

An online retailer employs AI-powered chatbots on their website to assist customers in finding suitable products. The chatbot analyzes user interactions and preferences to offer personalized recommendations, effectively collecting data on customer preferences and behavior.

Further consideration when collecting gamified data: - Ethical considerations: Avoid any manipulation, data exploitation or predatory approaches - Integration matters: Seamlessly blend research tasks with gameplay, avoid disruption, ensure the game itself is fun & engaging - Incentives, not just rewards: Use badges, recognition or progress to motivate, not just pay-to-do - Data quality checks: Validate responses, monitor for aberrations, assess completion rates and potential bias from the expected reward-seeking behaviour - Beyond just surveys: Explore gamified simulations, virtual environments or creative storytelling formats to unlock richer data and deeper insights (beyond just gamified surveys)

Whether you know it or not, we are all participating in gaming data collection. Think airline miles, points at hotels, super market rankings, Fortnite rewards, and so many others that follow certain patterns like these: -Achievement: achieve a badge, new level or points -Leader boards: create competition and more involvement -Schedule: time remaining to complete a task -Progress: display your success (and likely make you want more!!) Those are quick examples but all are used to collect customer data and improve the overall experience of the customer journey with a product or service. It’s important for the consumer to realize that their information is being accessed and used - but hopefully understand the end goal does help (them).

An educational app integrates quizzes and tasks into its interface, rewarding users with points and badges upon completion. This gamified approach incentivizes users to provide feedback on their learning experiences, resulting in higher engagement and reliable data.

IIt's not just about having a well-made gamified data collection system; timing is also important. Find a moment when your potential customer is bored, such as on an airplane, in line for their morning coffee, on a break at a conference, or on the way home from work. Provide related content, add a playful element, and then ask whatever you want. A bored person will give you their soul for entertainment.

Biometric approaches like eye tracking and EEG sensors allow for the capture of non-conscious emotional responses. This may reveal deeper insights beyond the stated feedback. For instance, eye tracking heatmaps optimize an ad's key elements. Computer vision techniques can also code facial expressions during usability testing or interviews to detect emotions and recognize deeper pain points. Natural language processing techniques enable the analysis of open-ended conversations. This automates text analysis without extensive coding. Leveraging such emerging techniques allows tapping into data not feasible through traditional collection methods. But they require new analytical expertise. The key is matching the method to the research goals.

Consumers have a right to know how their data is being used, and to opt out if they wish. Transparency and consent are not just legal requirements, they’re also key to maintaining trust with consumers.

Maintain constant engagement with your panel or partner panels. Confirm their information is used for surveys/research only.

In addition to the methods Jai mentioned, you can also detect emotion from voice signals. See my response to item #4 for more info about that, including some of its advantages.

A market research team experiments with virtual reality-based surveys, immersing respondents in simulated environments to collect feedback on experiences in an interactive manner, providing a novel way to gather insights beyond conventional methods.