How do you define the problem you want to solve with data?

- 📋 Identify the background, motivation, and goals to understand the context of the problem. - 🤝 Determine the stakeholders involved and their interests to align your solution with their needs. - 🚧 Define assumptions, constraints, and risks to set realistic boundaries for your analysis. - 📈 Establish clear success metrics to measure the effectiveness of your solution. - 🔍 Clarify the purpose and scope to focus on relevant data and avoid misleading conclusions.

To solve a problem with data, it's important to understand the context. This means knowing why you're looking at the data and what you hope to achieve. For example, if you're analyzing sales data to increase revenue, you need to understand the market trends, customer preferences, and seasonality. Knowing these details helps you make sense of the data and find useful patterns. Without context, it’s like trying to solve a puzzle without knowing what the finished picture looks like.

Para mapear o contexto do problema, eu levanto 5 critérios que chamo de ACERT: - Atores interessados (stakeholders) e seus interesses - Critérios de sucesso - Escopo - Restrições - Tempo para resolução Esses 5 critérios me direcionam para as perguntas que devo fazer ao dono do problema, para seguir para o próximo passo, o da estruturação. E depois realizar a coleta de dados mais direcionada.

Eu utilizo 5 critérios para entender o contexto de um problema, de forma estruturada. Dei o nome de metodologia ACERT, para facilitar: - Atores interessados: partes interessadas do problema e seus interesses - Critérios de Sucesso: qual o objetivo e meta para resolver esse problema - Escopo: qual o escopo do problema? - Restrições: há restrições tanto na solução do problema como no processo para chegar nela? - Tempo: qual o prazo para resolvê-lo? Mapeando esses 5 critérios, fica mais fácil de saber quais perguntas você deve fazer para ter um bom entendimento do contexto do problema.

To solve a problem using data, first identify the goal, understand the context, form the problem into manageable parts, define success metrics, identify data requirements, explore and prepare the data, choose appropriate analytical techniques, and use an iterative approach. Communicate the results in a clear and understandable manner, such as through visualizations, reports, or presentations, to stakeholders. This process ensures a comprehensive and effective solution to the problem.

One of the most important yet underrated skills in Data Analytics is the ability to ask the right questions. Getting to the right questions helps us land on the solution rather quickly and I cannot overemphasize the skill of coming up with hypothesis and looking for evidence in the form of data to prove or disprove them. Important to ensure we have a MECE and SMART qualified hypothesis

Defining the question is the first step in solving a problem with data. Start by thinking about what you want to find out or understand better. Keep it simple and clear. For example, if you're trying to improve sales, your question might be, "What factors influence customer buying decisions?" Make sure your question is specific and focused so that you can gather the right information and find meaningful answers.

Crafting a clear, actionable question is essential for effective data analysis. By defining a SMART goal, you provide clear direction for your data collection and analysis efforts. For instance, instead of asking "How can we improve website traffic?", specify "Can we increase website traffic by 20% through SEO optimization in the next six months?" This focused approach ensures your data directly addresses your desired outcome.

I applied the SMART criteria during our annual review to redefine our financial audit goals. Previously, our goals were too vague, like "reduce expenses". By rephrasing them to "cut operational costs by 15% within the fiscal year through targeted reductions in supply chain inefficiencies," we set a clear, actionable, and time-bound path that significantly improved our strategic approach and measurable outcomes.

On top of the problem statement being SMART, it's important to frame it in a way that's aligned with the organization's bottom-line goals and values. Consider its broader implications and how addressing it can drive strategic outcomes beyond the immediate scope. This holistic approach ensures that your data-driven efforts are not just effective but also impactful in driving meaningful change.

A lot of times the data might be readily available, but sometimes we might not have the right kind of data, in which case we might have to resort to some industry reports or other proxies that serve as a source for our data

Internal data sources like customer databases, sales records, and operational metrics offer insights into organizational performance and market trends. Previous research within the organization can also contribute. External sources, including government datasets and industry reports, provide context and insights into market dynamics and consumer preferences. Qualitative data sources like interviews and focus groups reveal rich insights into customer perceptions and behaviors, offering valuable context despite being less quantifiable. Quantitative sources, including internal databases and surveys, provide numerical data for statistical analysis, enabling rigorous testing of relationships and informed decision-making.

In my work, identifying the right mix of data sources has been key to accurate financial forecasting and budgeting. We combine internal sales data, industry benchmarks, and macroeconomic indicators to provide a comprehensive view that supports more robust decision-making. This multi-source approach has helped us anticipate market trends more effectively and align our strategies accordingly.

Determine where the relevant data resides. This might include internal databases, external sources, or data that needs to be collected through surveys, sensors, or other means. It's important to consider the quality, accessibility, and relevance of each potential data source.

Get creative and think beyond the obvious. Tap into unconventional data sources to gain unique insights. Collaborate with data scientists and industry experts to explore advanced integration techniques. Keep an eye on emerging data technologies like blockchain for secure data sharing or AI-driven tools for data enrichment. By exploring these avenues, you can ensure a comprehensive and innovative approach to sourcing the data you need for effective problem-solving.

Planning the data collection means deciding how to gather the information you need to answer your question. First, figure out what data you need and where you can find it. Then, think about how to collect it—whether it's through surveys, interviews, or existing databases. Make a detailed plan, including who will collect the data, when, and how often. Keep in mind any ethical or privacy concerns, and ensure that you're collecting enough data to get reliable results. Finally, be flexible and ready to adjust your plan as needed along the way.

Planning data collection is essential for efficient information gathering. Define needed data, choose methods (e.g., surveys, interviews), set timelines, allocate resources. Consider ethics, biases, data security. A structured plan ensures accurate, relevant data for effective analysis and problem-solving

Data collection is a crucial step in any of the project based on data. Because the collected data can highly influence the problem and direction of analysis. Hence it is very important to know and review multiple times what people will interpret on answering the questions. Also, while designing questions, researchers should focus on legal laws of data privacy, and level of understanding of audience.

For a major project aimed at optimizing our inventory management, I led the planning of a data collection strategy that involved both RFID tracking for real-time data and regular manual audits to ensure data accuracy. This planning ensured that the data collected was not only sufficient but also precise, leading to actionable insights that significantly reduced our inventory costs.

It's okay to follow a proven routine, but it's better to find ways to innovate the process. Fostering a culture of continuous improvement by regularly reviewing and refining your data collection methods ensures you can adapt to new challenges and opportunities and ultimately keep your process both robust and agile.

Reviewing your data collection process goes beyond just checking for accuracy and completeness - it's about continuously improving data quality. Consider using data visualization tools to quickly spot patterns, gaps, or outliers. This makes discrepancies easier to see and address. Also, fostering a culture of transparency and accountability in your team ensures everyone is committed to maintaining high data quality.

I would go into a little more detail. Reviewing the data collection process is about ensuring that the data you collect will be suitable for solving the defined problem. Assessing Data Sources: Identify where the required data can be sourced from, whether it's internal databases, public datasets, or data that needs to be collected through surveys or other methods. Evaluating Data Quality: Assess the quality of the data in terms of accuracy, completeness, timeliness, and relevance. Poor data quality can lead to incorrect conclusions Considering Data Privacy and Ethics: Ensure that data collection and usage comply with legal and ethical standards, especially when dealing with sensitive or personal information.

After gathering data, put it to the test. Check for accuracy, completeness, consistency, and relevance. Identify any gaps, errors, outliers, or biases – these can skew results. Document everything and prep your data for analysis.

While reviewing data collection, it is important to integrate "data provenance" and "lineage tracking" tools. These technologies trace the origins and transformations of the data throughout its lifecycle, ensuring transparency and accountability. For example, in financial analytics, tracking the lineage of transaction data helps verify its integrity and compliance with regulations. By implementing provenance and lineage tracking, organizations can more effectively identify and rectify data quality issues, understand the impact of data transformations, and maintain high standards of data governance, ultimately leading to more reliable and trustworthy analytics outcomes.

Evaluating the ethical implications of data analysis is critical to ensure that the process is responsible, compliant, and feasible. Some important points to consider, 1. Data Privacy: Ensure that the data used in analysis has been collected with informed consent and that privacy is maintained. Personal information should be anonymized. 2. Data Bias: Analyze whether the data or the algorithms used might introduce bias, leading to unfair or discriminatory outcomes. 3. Data Transparency : Maintain transparency in how data is collected, processed, and used for analysis. Establish clear accountability for the outcomes of the data analysis, ensuring that there are mechanisms in place to address any negative consequences.

When defining a problem to solve with data, it's essential to keep in mind the dynamic nature of data itself. Data is not static; it changes and evolves with time and circumstances. As such, your approach to defining and solving problems with data should be adaptable. Continuous monitoring and updating of data sources, as well as methodologies, are necessary to ensure that the solutions remain effective and relevant. Additionally, consider involving a diverse team in the problem definition process. Different perspectives can lead to a more comprehensive understanding of the problem at hand, which in turn can lead to more innovative and effective solutions.

When defining the problem you want to solve with data, take the bigger picture and potential ripple effects into account. Think about how your findings might impact different stakeholders and anticipate any unintended consequences. For example, if you're analyzing customer feedback to enhance a service, consider how those changes could impact other parts of the business, like operations or marketing.

Before delving into data research and analysis, it is crucial to first define the problem you aim to solve. Rushing into data analysis without a clear problem definition can lead to misguided efforts and inaccurate conclusions. Take the time to establish the context, set specific goals, identify stakeholders, and outline success metrics to ensure your data work is focused and effective.

It is important to have "human-in-the-loop" (HITL) systems in the data problem-solving process. HITL involves integrating human expertise into the AI and analytics workflow to improve model accuracy and decision-making. For instance, in fraud detection, algorithms can flag suspicious transactions, but human analysts review and validate these flags, providing nuanced insights that automated systems might miss. This collaborative approach combines the strengths of human intuition and machine efficiency, ensuring more robust and adaptable solutions that continuously improve with human feedback.