Critique of the advent of ChatGPT affecting the data analytic job market (UOG1124006)

Owolabi's paper, "The advent of ChatGPT: Job Made Easy or Job Loss to Data Analysts," raises the very important question about the role of AI in the future of data analysis. Their findings show that while a strong force, ChatGPT simply cannot replace an accomplished human analyst. This conclusion is both sensible and reassuring. The timeliness and empirical approach to testing the limits of AI in a highly specialized field alone merit notice. 

However, despite having the right question and a sound conclusion, there are substantial methodological and conceptual issues with this research that make it not very convincing as a contribution to the discussion. The main strength of this study is the hands-on approach. By showing direct conversational outputs from ChatGPT, the authors give a clear vision of current capabilities and limitations that the AI maintains. For readers who are not well-versed in data science, this walkthrough was enlightening-it serves to show quite clearly how dependent the model is on multiple prompts, how incapable it currently is of carrying out code it generates, and how ultimately reliant on human expertise it remains. This narrative format directly illustrates the core message that domain knowledge and critical thinking remain irreplaceable for now. However, the major weakness of the present study is the experimental design, in which it was closer to a single case study than a scientific experiment. The researchers used a simulated dataset with statistical problems such as multicollinearity and outliers intentionally embedded in it. Although such a setup guarantees control, it frames the task as a "textbook exercise" rather than what a data analyst actually does. In real-world data analysis, the challenge is that the data is messy and incomplete, and understanding the surrounding business environment is part of knowing the right questions to ask-complexities this study completely avoids.

Moreover, the quality of an LLM's output is heavily dependent on the quality of the "prompt engineering" used. The prompts in the study are extremely basic-for example, "Check this data out." An expert data analyst would use an order of magnitude more sophisticated and iterative prompting strategy, specifying libraries, suggesting analytical paths, and asking the model to check its work. The study tests a novice-level interaction and shows that a powerful tool yields poor results if used poorly, rather than truly investigating if ChatGPT can replace an expert analyst.

Conceptually, the paper also seems to misunderstand the intended workflow of using AI like ChatGPT for data analysis. The authors repeatedly highlight the model's inability to run code as a major limitation. That is a known characteristic of the technology; ChatGPT is a code generator not a code executor. It is designed to be a partner in a development environment where a human analyst runs, debugs, and validates the code. Criticizing it for not executing code is like blaming a cookbook for not cooking the meal.

Finally, the scope of the paper is extremely narrow, while the scope of its conclusions is very broad. The whole experiment is based on a single task, namely performing a multiple linear regression on a small dataset comprising 30 observations. A data analyst's role is quite diverse, and it widely includes data cleaning and exploratory analysis, advanced visualization, feature engineering, different machine learning models, and most importantly, communicating the findings to stakeholders. Making such a broad judgment about the whole profession based on its performance in one basic statistical technique is already a great overreach.

In other words, Owolabi , have the right question and a plausible answer but poor research to connect them. The paper is useful as an introductory demonstration for persons with little technical experience with the limitations of ChatGPT. Otherwise, the paper is lacking in methodological rigor and employs simplistic prompting, while its scope is limited, and the understanding of the tool's intended use superficial. The study confirms what many suspect-that expertise still matters-but does so through an investigation that does not hold up to critical scrutiny.

Researchgate.net. [Online]. Available: https://www.researchgate.net/profile/Abiola-Owolabi-2/publication/379586278_The_advent_of_ChatGPT_Job_Made_Easy_or_Job_Loss_to_Data_Analysts/links/6612d6ef3d96c22bc77acb83/The-advent-of-ChatGPT-Job-Made-Easy-or-Job-Loss-to-Data-Analysts.pdf.