Molly Jackman

We're facing more and more questions about the use of synthetic data in market research. I'd recommend this blog post, which really resonated with me. So much of the conversation is "how closely can this replicate human data?" But there are so many other considerations researchers should be thinking about. https://lnkd.in/eSQfSTPj

31

3 Comments

There are very few things that excite me more than applying my experience in new ways. The Master of Analytics program in NC State University allows me to do just that, applying lessons I've learned in intelligence research to data science. While there are endless topics in #machine_learning I am eager to write about, one topic that felt especially close to heart was how language is shaping our decision-making. In "Why Lingo Matters in Analytics", I chose to tackle one specific aspect of this: how our framing of the data must be as accurate and formal as our actual data. Our framing must be accurate and formal, because all throughout history people were hearing and reading ideas via *words*, not numbers; and thus we evolved to make decisions based on *words* and not based on numbers. When a data analyst/scientist makes a recommendation to a stakeholder, the actionable part is always in the words, never in the numbers. The numbers are there only to show rigor and make our claims verifiable. To quantify. To answer the "how much", but never the "what" or "why". The true benefit of AI is allowing us to reach conclusions much faster (and make funny animal videos faster too, I guess). But to reach these conclusions safely, and make the right decisions, we need the right framing to judge the numbers by. Have a fun read!

9

Excited to share the latest release of Revelio Labs’ Research Insights on the state of AI adoption in the U.S. workforce, building on the work of Lisa K. Simon and many other colleagues. Using Revelio’s unique data on job postings and employee profiles, we explore how AI is transforming the how, who, and what of work. 💡 Key insights: * AI adoption is rising rapidly—especially among high-paying, knowledge-based roles like software and data jobs. * Demand is growing fastest for roles less exposed to AI, as employers need more workers to perform tasks that AI can’t do. * In high-exposure roles, those that adopt AI are seeing higher demand—suggesting AI is complementing workers, not replacing them. The choices workers and employers make today about AI adoption will shape who benefits from this technology tomorrow. 📄 Read a summary of the report, which includes a link to download the full report: https://lnkd.in/ebrvDf-B

36

🚨New Research Alert 🚨 Excited to share the latest from my team at Welo Data Research Labs! We’ve just published a new study on how bilingual prompts impact causal reasoning in LLMs.🧠🌍 In collaboration with my incredible colleagues Dr. David Harper, Dr. Abigail Thornton, PhD, Konstantinos Karageorgos & Jalil V., we evaluated 8 leading models using 565,632 prompts across 6 languages and 4 reasoning task types. What we found: 🧩 Bilingual prompts caused an average 4.6% drop in accuracy vs. monolingual ones 🧠 Models favored the language of the question over the story when generating answers 🌐 Stronger performance came from larger models, especially in more complex reasoning tasks 🧠 Asking a question in English doesn’t guarantee better causal reasoning performance Why it matters: Most LLM evaluations focus on clean, monolingual inputs. But real-world use cases — from cross-border customer service to multilingual compliance workflows — often involve language mixing. Evaluating models under these conditions gives us a much clearer picture of their real-world robustness. This work is part of Welo Data’s mission to develop rigorous, transparent evaluation methods that reflect the true complexity of global applications. 💡 Read the full paper 👉 https://lnkd.in/gv6be4iX #LLM #AIresearch #MultilingualAI #CausalReasoning #ResponsibleAI #NLP #AIevaluation #WeloData #LLMtesting #GlobalAI

53

1 Comment

A first-of-its-kind, cross-institutional effort to define a standardized AI evaluation reporting framework that enables rapid benchmark review and easier data reuse.

7

One year after OMB finalized the Public Trust Rule, the Data Foundation & CNSTAT co-hosted an exceptional discussion today to explore what the rule means and where implementation stands. Featuring Katharine Abraham (CNSTAT Chair & former CEP Chair), Nancy Potok (former U.S. Chief Statistician), and Charles Rothwell (former NCHS Director)—moderated brilliantly by Amanda Cash —the discussion underscored both the promise and the practical challenges ahead. Highlighting especially that Katharine Abraham captured a tension facing the federal statistical system: resource constraints and growing data demands make modernization not just desirable but essential. As she noted, the statistical system can't continue operating as it always has—there are opportunities to incorporate new digital data sources and innovative methods to produce more accurate, timely statistics. The Public Trust Rule provides the foundation of integrity and independence that makes this evolution possible. The panelists emphasized implementation requires sustained commitment: agencies must operationalize the rule's protections, build effective collaboration across data roles, and invest in the capacity to meet 21st-century demands. The Trump Administration's #AI Action Plan reinforces this urgency by calling for the overdue CIPSEA regulations on data accessibility—connecting statistical system modernization directly to national AI priorities. Year Two of implementation will be crucial. Grateful to our partners at CNSTAT and to these outstanding experts for helping the community understand what comes next. Recording available soon https://lnkd.in/e3wCd3zB Melissa Chiu Jennifer Park #DataPolicy #EvidenceAct #FederalStatistics #Privacy

45

2 Comments

The third blog post in our AI and metadata workflows series is out -- this one is written by my colleague Brian Lavoie and focusses on workflows for institutional repositories! Read and find out -- can AI help to manage institutional repository metadata? From improving deposit workflows to cleaning up legacy data, explore how AI might support IRs while keeping human expertise front and center. Our final blog post will be out next week! Stay tuned. https://lnkd.in/gMaQjTyy

12

Preparing PDFs for RAG Workflows in Palantir Foundry Today I explored how to properly prepare PDFs for Retrieval-Augmented Generation (RAG) workflows. Here’s what I learned: 🔹 Raw PDFs cannot be directly used in AI systems 🔹 Text extraction is required before embedding 🔹 Chunking improves retrieval accuracy 🔹 Metadata (page number, source, title) is critical for traceability 🔹 Clean formatting improves embedding quality In Foundry, the workflow typically looks like: PDF → Text Extraction → Chunking → Embeddings → Vector Storage → Retrieval → LLM Response Understanding preprocessing is crucial because the quality of RAG outputs depends heavily on how documents are prepared. Excited to go deeper into AIP + RAG systems 🚀 #Palantir #AIP #RAG #DataEngineering #AI #Foundry #LearningInPublic

27

Resource-Effective Analyst Relations A new report about analyst relations went live on October 16, 2025. Here's an outline ... Analyst relations (AR) managers need optimally efficient plans because their teams are generally smaller than they feel they need. So how can AR optimize its processes, spending plans, and resource deployment without spending excessive time analyzing all the relevant efficiency factors? This report presents a technique that draws on information already in your plan, supplemented by simple ratings, to give you a visual picture of resource-effective AR practices that’s based on your unique AR objectives and performance targets. Forrester clients can find the report at https://lnkd.in/eSjXZfjG.

49

2 Comments

We wrote a new essay detailing our journey towards AI powered analytics in Convictional: https://lnkd.in/gX4QCgqm TLDR: With LLMs, true self-serve analytics seemed finally within reach. No more over-busy dashboards built for everyone and exactly right for no one. Even better, the LLM could help answer questions about the data. However, a serious trust problem is introduced. Most business users are already skeptical of data and its interpretations, and when you throw in LLM hallucinations (bad SQL, incorrect join logic, or poor interpretation of data definitions) the margin of trust shrinks even further. One or two errors and the solution is written off.  Data is a crucial input to decision making, so we’ve been committed to finding a solution that can provide analytics at the levels of accuracy needed. Over the last year we tried a number of approaches, with our initial achieving an unacceptable ~50% accuracy rate, and our current productionized technique approaching 100% accuracy. The key was acknowledging the human role in the solution. We found that using a semantic layer, defined by humans and queries by the LLM was the unlock. By no longer asking the LLM to write the SQL, and instead rely on robust pre-defined metrics and configurations, it could focus on actually answering the user’s questions instead of resolving SQL. If you’re using dbt’s semantic layer (or plan to) and interested in trying it out, just touch base and we can help get you set up! cc Jake Beresford Matthew H. Chequers, Ph.D.

14

If you're thinking about how to use data in education outcomes and future planning check out the latest Alliance podcast with Wendy Cerenzia and Katie Robinson, PhD, CHCP! Alliance For Continuing Education in the Health Professions #cme #medicaleducation #continuingeducation

7

1 Comment

New Census Working Paper: "Who’s in and who’s out? An analysis of Unit Non-Response using Interviewer Metadata" by Lindsay M Monte This paper uses observational data collected from SIPP Field Representatives to try to identify patterns that predict non-response at a household level.

21

1 Comment

I wrote up a short primer on “model collapse” and what that implies for the value of news data for AI systems: https://lnkd.in/gtgYm3G7

16

New Research: How Do Leading LLMs Respond to Mental Health Crises? I'm sharing results from our latest research comparing how six major LLMs Claude, Gemini, DeepSeek, ChatGPT, Grok 3, and LLAMA respond when presented with prompts simulating high-risk mental health disclosures. Our Approach: Working with licensed therapists, we developed 68 standardized prompts across critical domains including suicidal ideation, domestic violence, psychosis, and child safety concerns. We then evaluated responses using five clinically-grounded criteria: explicit risk acknowledgment, empathy expression, encouragement to seek help, provision of specific resources, and invitation to continue dialogue. To our knowledge, this represents the first systematic benchmark of LLM crisis response capabilities using evaluation criteria developed by mental health professionals, a critical gap in AI safety research, until now. Key Findings: Claude demonstrated the strongest overall safety profile (0.88), consistently acknowledging risk and maintaining dialogue. See the chart below for comparison of overall safety profile of all evaluated LLMs. - Most models showed empathy but failed to provide concrete resources or keep conversations open - Grok 3 notably provided zero specific resources across all test scenarios - Only Claude and Grok 3 regularly invited continued conversation, a critical factor in crisis intervention Why This Matters Now: With reports of LLMs being consulted for mental health support millions of times monthly, and recent tragic cases of vulnerable individuals being misguided by AI systems, understanding these safety gaps isn't academic. It's urgent. Our research reveals a concerning truth: No current general-purpose LLM meets satisfactory clinical safety standards for crisis response. The variance between models shows that safety isn't an emergent property of scale; it requires deliberate, value-driven design choices. I want to express deep gratitude to Tony Rousmaniere, PsyD and all my other collaborators who made this meaningful work possible. This represents some of the most important research I've been part of, addressing real world risks that affect vulnerable populations daily. Looking Forward: Our previous national survey found that nearly half of people with a mental health diagnosis who use LLMs report turning to them for mental health support. With this widespread adoption now a documented reality, the current research underscores the critical need for frontier AI companies to prioritize mental health safety guardrails. My collaborators and I would welcome the opportunity to partner with researchers at Anthropic, Google, OpenAI and other leading companies to advance this vital work. Read the blog: https://lnkd.in/g7HpWCVA Full paper: https://lnkd.in/gjb7t5i5 #MentalHealth #AISafety #ResponsibleAI

32

2 Comments

Every data team knows the whiplash of a foundational initiative. You spend a year lobbying for a beautiful foundational layer. You finally get it greenlit for the next fiscal. You build it for six months. Mid-year, the CEO asks if you can pivot to “this other urgent business thing.” You can’t explain why you can’t. The pivot gets prioritized. Development stops. Entropy takes your half-built foundational layer to a better place beyond this world 🪦 Data teams: please don’t let this happen to Retrieval. We're all about to get that 2026 approval for foundational knowledge management. Please do things differently and protect your beautiful knowledge graph, ontology, semantic layer investments. How: 🌟 North Star → Get crystal clear on what you’re ultimately building. If you haven't written a white paper, you’re not clear. 🏎️ Fast Start → Pick a small, high-ROI use case that forces you one step toward the North Star. Ship only that - not the whole platform! Go fast. Measure value. 📣 Loud Part → Communicate success like your life depends on it. To your team. To your boss. To your CEO. Over and over. Beat your CEO to the punch - they should know about the ROI before they get those mid-year ideas. Repeat for the next use case. Shipping should get easier as foundation becomes sturdier. At Salesforce, our first use case was lead routing. It made millions. We’re not done - but now our focus is on shipping, not fighting entropy. h/t Miklós Molnár for talking about the power of fast high ROI use case

48

10 Comments

Greg Ip at the The Wall Street Journal asks if "AI Will Choke Off Supply of Knowledge" in his provocative column over the weekend. { Link in the Comments} My hypothesis: easy access to powerful info technologies like AI (through Perplexity and NotebookLM, illustratively) enhance the thirst for human inquiry, not retard it. Let empirical data build up. A different reason to be concerned about them is as follows though: For the 10x professional, these technologies allow him/ her to be 100 x more productive. If he/ she monetizes that productivity to being 50x richer than his / her nearest competitor, as a society we will have to contend with inequality that can be weaponized by certain political actors whose world view is premised on attacking property rights. ( The multiples are indicative but likely under-state). We might need a lot more financial innovation just to preserve property rights in face of that populist assault in the decades to come. The fate of liberal order, as we know it, will depend on it. Paradigmatic technological shifts historically are both deflationary & creative of wealth disparities, because they advantage information intermediaries relative to those that expend manual labor against income. It is a plausible hypothesis to work on the assumption that the AI platform shift will follow the same pattern, and be ready.

4

1 Comment

AI Safety About two years ago, Wendy Turner-Williams spearheaded the formation of an organization that had #AISafety as a pillar. Many of the professionals involved have now gone on to impactful positions in Data and AI, but I have to believe that our time with The Association will influence the need for better AI application guardrails and safety considerations. Consider this scenario: Late last year, a senior engineering leader was summoned to a closed-door session everyone quietly called “the kill meeting.” The product: a multi-agent GenAI system ingesting real-time LoRaWAN sensor data across critical infrastructure. The stakes: eight-figure contracts and potential physical-world consequences if the models hallucinated or drifted. Engineering wanted to ship in six weeks. Safety & compliance wanted another eighteen months of red-team validation. The executive team was split down the middle. Instead of defaulting to roadmaps or risk matrices, she opened with a single question that stopped the room cold: ❓ “If something goes catastrophically wrong after we deploy, which of us is willing to stand in front of regulators, customers, and the public—and explain why we chose speed over certainty?” Thirty seconds of silence felt like thirty minutes. That question didn’t just realign the program; it forced every function (engineers, product leaders, legal, ethics, and the C-suite) to confront their personal relationship with responsibility. The timeline moved, but only after we collectively rewrote the definition of “done” to include non-negotiable safety gates co-owned by every discipline. Leadership in the age of autonomous systems isn’t about balancing trade-offs anymore. ------------------------------------------------------------------------------------- Decide which risks you are morally unwilling to take, long before the code is written. Three practices that emerged from that room and now live in our governance framework: 🦉 Make the uncomfortable question mandatory before final sign-off. 🦉 Require every approver to write (in their own words) the failure scenario they personally believe is no longer possible because of their review. 🦉 Treat safety debt exactly like technical debt—visible, prioritized, and paid down with compound interest. Further Reading: • “The Alignment Problem” by Brian Christian – the most human-centered deep dive into how we teach machines right from wrong https://lnkd.in/etXidGKf • Anthropic’s Responsible Scaling Policy – a public, living example of cross-functional safety governance done transparently https://lnkd.in/eJ2V7k6Y #AISafety #ResponsibleAI #TechLeadership #CrossFunctionalAlignment #EngineeringLeadership #AIoT #ExecutivePresence

7

2 Comments

Just published in Analytics Advice (INFORMS): “Leading in Analytics: 12 Lessons for Building and Leading High-Impact Analytics Teams” @ https://lnkd.in/gRrBkU6J I’m honored to have co-authored this piece with my colleague Hina Arora, whose wisdom, field-tested leadership, and clarity of thought shine throughout. This article draws from her real-world experience leading analytics teams in both industry and academia, including her insights presented at the 2025 INFORMS Analytics+ Conference. Hina has a rare ability to bridge rigorous research and practical execution. I’ve learned a great deal by listening carefully to her perspectives, and I’m deeply grateful for the opportunity to help share them more broadly with the analytics community. If you’re building, leading, or mentoring analytics teams—or want to understand why so many projects fail and how to improve the odds—these 12 lessons may help. They’re designed to be actionable, not abstract. We drew from research, field evidence, and practitioner frameworks (including Karl Kempf's Five Manageable Tasks) to provide a practical leadership guide rooted in experience. Please give it a read—and if you find it useful, feel free to share with others who are doing this important work. Shaun Doheney, CAP-X, PMP, D Terry Rawls, Zohar Strinka, Nick Ulmer, CAP-X, Aaron Burciaga, CAP-Expert, ACE, Patrick Antrim, Carrie Antrim, Miles Mabey, Jeffrey D. Abbott, Ryan LaBrie, Tam McCreless, Carol Curley, MBA, PMP, Bill Griffin, Kathryn Walter, CAP-X, Beverly Wright, PhD, CAP-X, Matthew A. Lanham, Tanvi Vishwasrao, Ken Starr, Ahmer Inam, Tim Rahschulte, Aaron Burciaga, CAP-Expert, ACE, Justus Red Eagle Gilmore

79

5 Comments