Data Skeptic

Kyle Polich

About

The Data Skeptic Podcast features interviews and discussion of topics related to data science, statistics, machine learning, artificial intelligence and the like, all from the perspective of applying critical thinking and the scientific method to evaluate the veracity of claims and efficacy of approaches.

Available on

Community

525 episodes

Advances in Data Loggers

Our guest in this episode is Ryan Hanscom. Ryan is a Ph.D. candidate in a joint doctoral evolution program at San Diego State University and the University of California, Riverside. He is a terrestrial ecologist with a focus on herpetology and mammalogy.  Ryan discussed how the behavior of rattlesnakes is studied in the natural world, particularly with an increase in temperature.

35m
Mar 25
What You Know About Intelligence is Wrong (fixed)

We are joined by Hank Schlinger, a professor of psychology at California State University, Los Angeles. His research revolves around theoretical issues in psychology and behavioral analysis.  Hank establishes that words have references and questions the reference for intelligence. He discussed how intelligence can be observed in animals. He also discussed how intelligence is measured in a given context.

41m
Mar 20
Animal Decision Making

On today’s episode, we are joined by Aimee Dunlap. Aimee is an assistant professor at the University of Missouri–St. Louis and the interim director at the Whitney R. Harris World Ecology Center. Aimee discussed how animals perceive information and what they use it for. She discussed the connection between their environment and learning for decision-making. She also discussed the costs required for learning and factors that affect animal learning.

37m
Mar 12
Octopus Cognition

We are joined by Tamar Gutnick, a visiting professor at the University of Naples Federico II, Napoli, Italy. She studies the octopus nervous system and their behavior, focusing on cognition and learning behaviors. Tamar gave a background to the kind of research she does — lab research. She discussed some challenges with observing octopuses in the lab. She discussed some patterns observed by the octopus lifestyle in a controlled setting. Tamar discussed what they know about octopus intelligence. She discussed the octopus nervous system and why they are unique compared to other animals. She discussed how they measure the behavior of octopuses using a video recording and a logger to track brain activity.

38m
Mar 08
Optimal Foraging

Claire Hemmingway, an assistant professor in the Department of Psychology and Ecology and Evolutionary Biology at the University of Tennessee in Knoxville, is our guest today. Her research is on decision-making in animal cognition, focusing on neotropical bats and bumblebees. Claire discussed how bumblebees make foraging decisions and how they communicate when foraging. She discussed how they set up experiments in the lab to address questions about bumblebees foraging. She also discussed some nuances between bees in the lab and those in the wild. Claire discussed factors that drive an animal's foraging decisions. She explained the foraging theory and how a colony works together to optimize its foraging. She also touched on some irrational foraging behaviors she observed in her study. Claire discussed some techniques bees use to learn from past behaviors. She discussed the effect of climate change on foraging bees' learning behavior. Claire discussed how bats respond to calling frogs when foraging. She also spoke about choice overload in that they make detrimental decisions when loaded with too many options.

38m
Feb 28
Memory in Chess

On today’s show, we are joined by our co-host, Becky Hansis-O’Neil. Becky is a Ph.D. student at the University of Missouri, St Louis, where she studies bumblebees and tarantulas to understand their learning and cognitive work.   She joins us to discuss the paper: Perception in Chess. The paper aimed to understand how chess players perceive the positions of chess pieces on a chess board. She discussed the findings paper. She spoke about situations where grandmasters had better recall of chess positions than beginners and situations where they did not.   Becky and Kyle discussed the use of chess engines for cheating. They also discussed how chess players use chunking. Becky discussed some approaches to studying chess cognition, including eye tracking, EEG, and MRI.  ## Paper in Focus Perception in chess ## Resources Detecting Cheating in Chess with Ken Regan

48m
Feb 12
OpenWorm

On this episode, we are joined by Stephen Larson, the CEO of MetaCell and an affiliate of the OpenWorm foundation. Stephen discussed what the Openworm project is about. They hope to use a digital  ( for short) to study the basics of life. Stephen discussed why  is an ideal organism for studying life in the lab. He also discussed the steps involved in simulating a digital organism. He mentioned the constraints on the cellular scale that informed their development of a digital . Stephen discussed the validation process of the simulation. He discussed how they discovered the best parameters to capture the behavior of natural . He also discussed how biologists embraced the project. Stephen discussed the computational requirements for improving the simulation parameters of the model and the kind of data they require to scale up. Stephen discussed some findings that the machine-learning communities can take away from the project. He also mentioned how students can get involved in the Openworm project. Rounding up, he shared future plans for the project.

34m
Feb 05
What the Antlion Knows

Our guest is Becky Hansis-O’Neil, a Ph.D. student at the University of Missouri, St Louis, and our co-host for the new "Animal Intelligence" season. Becky shares her background on how she got into the field of behavioral intelligence and biology.

41m
Jan 30
AI Roundtable

Kyle is joined by friends and former guests Pramit Choudhary and Frank Bell to have an open discussion of the impacts LLMs and machine learning have had in the past year on industry, and where things may go in the current year.

50m
Jan 17
Uncontrollable AI Risks

We are joined by Darren McKee, a Policy Advisor and the host of Reality Check — a critical thinking podcast. Darren gave a background about himself and how he got into the AI space. Darren shared his thoughts on AGI's achievements in the coming years. He defined AGI and discussed how to differentiate an AGI system. He also shared whether AI needs consciousness to be AGI. Darren discussed his worry about AI surpassing human understanding of the universe and potentially causing harm to humanity. He also shared examples of how AI is already used for nefarious purposes. He explored whether AI possesses inherently evil intentions and gave his thoughts on regulating AI.

38m
Dec 27, 2023
I LLM and You Can Too

It took a massive financial investment for the first large language models (LLMs) to be created.  Did their corporate backers lock these tools away for all but the richest?  No.  They provided comodity priced API options for using them.  Anyone can talk to Chat GPT or Bing.  What if you want to go a step beyond that and do something programatic?  Kyle explores your options in this episode.

23m
Dec 23, 2023
Q&A with Kyle

We celebrate episode 1000000000 with some Q&A from host Kyle Polich.  We boil this episode down to four key questions: 1) How do you find guests 2) What is Data Skeptic all about? 3) What is Kyle all about? 4) What are Kyle's thoughts on AGI?   Thanks to our sponsors dataannotation.tech/programmers https://www.webai.com/dataskeptic  

40m
Dec 19, 2023
LLMs for Data Analysis

In this episode, we are joined by Amir Netz, a Technical Fellow at Microsoft and the CTO of Microsoft Fabric. He discusses how companies can use Microsoft's latest tools for business intelligence. Amir started by discussing how business intelligence has progressed in relevance over the years. Amir gave a brief introduction into what Power BI and Fabric are. He also discussed how Fabric distinguishes from other BI tools by building an end-to-end tool for the data journey. Amir spoke about the process of building and deploying machine learning models with Microsoft Fabric. He shared the difference between Software as a Service (SaaS) and Platform as a Service (PaaS). Amir discussed the benefits of Fabric's auto-integration and auto-optimization abilities. He also discussed the capabilities of Copilot in Fabric. He also discussed exciting future developments planned for Fabric. Amir shared techniques for limiting Copilot hallucination.

29m
Dec 12, 2023
AI Platforms

Our guest today is Eric Boyd, the Corporate Vice President of AI at Microsoft. Eric joins us to share how organizations can leverage AI for faster development. Eric shared the benefits of using natural language to build products. He discussed the future of version control and the level of AI background required to get started with Azure AI. He mentioned some foundational models in Azure AI and their capabilities. Follow Eric on LinkedIn to learn more about his work. Visit today's sponsor at https://webai.com/dataskeptic

33m
Dec 04, 2023
Deploying LLMs

We are excited to be joined by Aaron Reich and Priyanka Shah. Aaron is the CTO at Avanade, while Priyanka leads their AI/IoT offering for the SEA Region. Priyanka is also the MVP for Microsoft AI. They join us to discuss how LLMs are deployed in organizations.

35m
Nov 27, 2023
A Survey Assessing Github Copilot

In this episode, we are joined by Jenny Liang, a PhD student at Carnegie Mellon University, where she studies the usability of code generation tools. She discusses her recent survey on the usability of AI programming assistants. Jenny discussed the method she used to gather people to complete her survey. She also shared some questions in her survey alongside vital takeaways. She shared the major reasons for developers not wanting to us code-generation tools. She stressed that the code-generation tools might access the software developers' in-house code, which is intellectual property. Learn more about Jenny Liang via https://jennyliang.me/  

26m
Nov 20, 2023
Program Aided Language Models

We are joined by Aman Madaan and Shuyan Zhou. They are both PhD students at the Language Technology Institute at Carnegie Mellon University. They join us to discuss their latest published paper, PAL: Program-aided Language Models. Aman and Shuyan started by sharing how the application of LLMs has evolved. They talked about the performance of LLMs on arithmetic tasks in contrast to coding tasks. Aman introduced their PAL model and how it helps LLMs improve at arithmetic tasks. He shared examples of the tasks PAL was tested on. Shuyan discussed how PAL’s performance was evaluated using Big Bench hard tasks. They discussed the kind of mistakes LLMs tend to make and how the PAL’s model circumvents these limitations. They also discussed how these developments in LLMS can improve kids learning. Rounding up, Aman discussed the CoCoGen project, a project that enables NLP tasks to be converted to graphs. Shuyan and Aman shared their next research steps. Follow Shuyan on Twitter @shuyanzhxyc. Follow Aman on @aman_madaan.

32m
Nov 13, 2023
Which Programming Language is ChatGPT Best At

In this episode, we have Alessio Buscemi, a software engineer at Lifeware SA. Alessio was a post-doctoral researcher at the University of Luxembourg. He joins us to discuss his paper, A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages.  Alessio shared his thoughts on whether ChatGPT is a threat to software engineers. He discussed how LLMs can help software engineers become more efficient.

40m
Nov 06, 2023
GraphText

On the show today, we are joined by Jianan Zhao, a Computer Science student at Mila and the University of Montreal. His research focus is on graph databases and natural language processing. He joins us to discuss how to use graphs with LLMs efficiently.  

32m
Oct 31, 2023
arXiv Publication Patterns

Today, we are joined by Rajiv Movva, a PhD student in Computer Science at Cornell Tech University. His research interest lies in the intersection of responsible AI and computational social science. He joins to discuss the findings of this work that analyzed LLM publication patterns. He shared the dataset he used for the survey. He also discussed the conditions for determining the papers to analyze. Rajiv shared some of the trends he observed from his analysis. For one, he observed there has been an increase in LLMs research. He also shared the proportions of papers published by universities, organizations, and industry leaders in LLMs such as OpenAI and Google. He mentioned the majority of the papers are centered on the social impact of LLMs. He also discussed other exciting application of LLMs such as in education.

28m
Oct 23, 2023
Do LLMs Make Ethical Choices

We are excited to be joined by Josh Albrecht, the CTO of Imbue. Imbue is a research company whose mission is to create AI agents that are more robust, safer, and easier to use. He joins us to share findings of his work; Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safety.  

29m
Oct 16, 2023
Emergent Deception in LLMs

On today’s show, we are joined by Thilo Hagendorff, a Research Group Leader of Ethics of Generative AI at the University of Stuttgart. He joins us to discuss his research, Deception Abilities Emerged in Large Language Models. Thilo discussed how machine psychology is useful in machine learning tasks. He shared examples of cognitive tasks that LLMs have improved at solving. He shared his thoughts on whether there’s a ceiling to the tasks ML can solve.

27m
Oct 09, 2023
Agents with Theory of Mind Play Hanabi

Nieves Montes, a Ph.D. student at the Artificial Intelligence Research Institute in Barcelona, Spain, joins us. Her PhD research revolves around value-based reasoning in relation to norms. She shares her latest study, Combining theory of mind and abductive reasoning in agent‑oriented programming.

38m
Oct 02, 2023
LLMs for Evil

We are joined by Maximilian Mozes, a PhD student at the University College, London. His PhD research focuses on Natural Language Processing (NLP), particularly the intersection of adversarial machine learning and NLP. He joins us to discuss his latest research, Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities.

26m
Sep 25, 2023
The Defeat of the Winograd Schema Challenge

Our guest today is Vid Kocijan, a Machine Learning Engineer at Kumo AI. Vid has a Ph.D. in Computer Science at the University of Oxford. His research focused on common sense reasoning, pre-training in LLMs, pretraining in knowledge-based completion, and how these pre-trainings impact societal bias. He joins us to discuss how he built a BERT model that solved the Winograd Schema Challenge.

31m
Sep 11, 2023
LLMs in Social Science

Today, We are joined by Petter Törnberg, an Assistant Professor in Computational Social Science at the University of Amsterdam and a Senior Researcher at the University of Neuchatel. His research is centered on the intersection of computational methods and their applications in social sciences. He joins us to discuss findings from his research papers, ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning, and How to use LLMs for Text Analysis.

34m
Sep 04, 2023
LLMs in Music Composition

In this episode, we are joined by Carlos Hernández Oliván, a Ph.D. student at the University of Zaragoza. Carlos’s interest focuses on building new models for symbolic music generation. Carlos shared his thoughts on whether these models are genuinely creative. He revealed situations where AI-generated music can pass the Turing test. He also shared some essential considerations when constructing models for music composition.

33m
Aug 28, 2023
Cuddlefish Model Tuning

Hongyi Wang, a Senior Researcher at the Machine Learning Department at Carnegie Mellon University, joins us. His research is in the intersection of systems and machine learning. He discussed his research paper, Cuttlefish: Low-Rank Model Training without All the Tuning, on today’s show. Hogyi started by sharing his thoughts on whether developers need to learn how to fine-tune models. He then spoke about the need to optimize the training of ML models, especially as these models grow bigger. He discussed how data centers have the hardware to train these large models but not the community. He then spoke about the Low-Rank Adaptation (LoRa) technique and where it is used. Hongyi discussed the Cuttlefish model and how it edges LoRa. He shared the use cases of Cattlefish and who should use it. Rounding up, he gave his advice on how people can get into the machine learning field. He also shared his future research ideas.

27m
Aug 21, 2023
Which Professions Are Threatened by LLMs

On today’s episode, we have Daniel Rock, an Assistant Professor of Operations Information and Decisions at the Wharton School of the University of Pennsylvania. Daniel’s research focuses on the economics of AI and ML, specifically how digital technologies are changing the economy. Daniel discussed how AI has disrupted the job market in the past years. He also explained that it had created more winners than losers. Daniel spoke about the empirical study he and his coauthors did to quantify the threat LLMs pose to professionals. He shared how they used the O-NET dataset and the BLS occupational employment survey to measure the impact of LLMs on different professions. Using the radiology profession as an example, he listed tasks that LLMs could assume. Daniel broadly highlighted professions that are most and least exposed to LLMs proliferation. He also spoke about the risks of LLMs and his thoughts on implementing policies for regulating LLMs.

38m
Aug 15, 2023
Why Prompting is Hard

We are excited to be joined by J.D. Zamfirescu-Pereira, a Ph.D. student at UC Berkeley. He focuses on the intersection of human-computer interaction (HCI) and artificial intelligence (AI). He joins us to share his work in his paper, Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts.  The discussion also explores lessons learned and achievements related to BotDesigner, a tool for creating chat bots.

48m
Aug 08, 2023