This is an excellent article that mirrors my thoughts as well. I have used LLMs for data analysis for the past six months and can see the benefits and the drawbacks. Recently, I have not been able to use the plug-in Advanced Data Analysis due to endless network errors using GPT-4. Some Python packages such as SMOTE are not available. Most importantly, a user should have at least intermediate data science knowledge to write intelligent prompts, understand the necessary data wrangling steps such as EDA, and troubleshoot issues. As pointed out, you need a data scientist in the loop.