Data at Depth 11: Diversity, Adversity, Case Study With GPT-4 and External Validation
Sowing, growing, and showing across multiple social media platforms
Today, a few topics:
My Data at Depth: documenting my own creator journey - my raw and exposed data for the past 2 weeks. A definite uptick on Substack, a boost on my first article in Towards Data Science, and an uptick on LinkedIn, one last kick at Twi-X.
Thinking and Creating: Some tidbits on what I have been creating and doing during the past 2 weeks (7 new articles, and another Medium Boost).
Doing - Case Study on GPT-4 and External Validation: here’s why I don’t take an unreliable LLM like GPT-4 at its word. How I use an external tool to validate GPT-4 for article title generation
Let’s get to the details!
1. My Data at Depth - My Creator Journey
Who doesn’t like to see someone else’s numbers? What’s doing well for me these past two weeks? What has slowed down? Am I making progress.
The word of the past two weeks for this writer/creator is “diversification”.
I’ve seen progress across all the platforms that I produce on (except Twitter-X).
My Substack:
Substack growth for the past 2 weeks has been much better than the previous two weeks.
Last newsletter I reached 200 subscribers. For the past 2 weeks, I have 26 new subscribers including 1 new paid subscriber (thank you). All your support makes this work all worthwhile - VERY much appreciated.
I have also seen a huge increase in views and reads.
The numbers:
A lot of green on the above screenshot again - the number that stands out the most is that there are almost 1000 more viewers than the previous 30 days. This is a big jump (awesome!).
My Medium:
So far April has been a bit slow. My “other” regular job has been wrapping up for the semester so it involved some additional time for grading, student interaction, etc. So far in April:
I’m on about the same pace as last month. The bright spot for me is the article I was able to get published in Towards Data Science that was also boosted. You can see the spike in views and reads.
You can click above to view the article. This is a big step forward for me - to have an article accepted by a publication that has 700K followers, and then for the article to be recognized as good enough to be boosted.
I will be spending more time on creating a few more articles similar in style and content to submit to TDS over the next few weeks.
My followers have slowed down from the big surge in March/February. I am on track for about 1200-1500 this month.
LinkedIn:
I had a much better 2 weeks on LinkedIn - a few of my articles went northward of 2K impressions, leading to the best numbers I’ve seen in awhile:
If you’re keeping score, this is almost 10K more than the previous 2 weeks. I am definitely trending upwards here. I also have had a big (for me) uptick in followers - 85 in the past 2 weeks.
My Twitter (X):
Yes, I have decided to invest some time (and money) into Twitter-X. This is where I am encountering adversity. I have not had success on this platform.
However, after watching and listening to writing guru Tim Denning, I have decided to put some time into Twitter. And to stay accountable, I am going to document my progress on this newsletter.
I am starting with 505 followers on X as of today (Apr 18th).
Step 1 has been to sign up for a Premium account - to minimize all the noise and garbage that can be overwhelming on this platform.
My current stats are absolutely terrible:
Nowhere to go but up, right? I am formulating a plan and will stick with it over the next few months (at least). Wish me luck!
And so goes the fickleness and volatility of social media that the modern creator must deal with daily (even hourly).
And yet on we go!
2. Thinking and Creating
For the past 2 weeks I have had one good week of writing and one week where the demands of my regular job became fairly heavy (end of semester responsibilities).
My big accomplishment for these 2 weeks was having an article published in Towards Data Science (TDS) and then having this article boosted by Medium. Although the monetary return wasn’t super-high, it still felt really good to have these two things happen.
Definitely a validation of my writing and a sign that I am heading in the right direction.
With this in mind, I will be spending a good part of the next 2 weeks focusing on articles of similar style and content for further submissions to TDS.
Since my last newsletter update (Apr 4) I have published/submitted 7 articles.
3 articles are in the realm of AI, GPT-4, and Python:
Simple Streamlit Sliders: Intaractive Map Visuals with GPT-4 Prompting
The Art of Prompting GPT-4: Python CSV Cleaning and Data Visual Code
Why GPT-4 Data Isn’t As Reliable As You Think
And 4 in the domain of in-depth data analysis/data storytelling:
Why Human-Centred Approaches Lead to Better Algorithm Design (Boosted!)
What the Heck is Data Storytelling Anyways? Here Are The Basics
How Multi-functioning Data Visuals Provide Deeper Data Storytelling (5000+ impressions on LinkedIn)
Why It’s a Great Idea To Write For Data Storytelling Corner (a small publication on Medium that I am an editor for)
These past 2 weeks, I spent more time than I usually do reading and researching. My volume is down a little bit due to my work and also on the time I am spending learning about Twitter (X) and how to leverage the platform to reach more of the 500 million users that are out there.
For the next two weeks I have some focal points for my writing and researching:
Learning more about Twitter (X) - How to leverage my new Premium account to reach out to more folks who have similar interests and ideas
Writing 1-2 articles that are relevant for publication in TDS (on algorithm thinking and design)
Thinking and research on course materials that I can add that are relevant to the audiences that I have (I am open to suggestions!).
More outreach and engagement on Substack to grow my audience here.
With my regular work wrapping up, I will have more focussed time to work on these tasks.
3. Case Study: Combining GPT-4 With External Validation For Effective Results
For this section, I want to change the focus slightly from what I have been publishing recently. Today, I am going to show you a case study on how I use GPT-4 to assist me in my writing workflow - and with quantifiably great results.
Since I started spending more time on crafting proper titles, I have increased my Medium article Boost rate from 2% to 30%.
GPT-4 now helps me with title generation for my articles. But not on its own.
As many researchers have shown, GPT-4 is unpredictable and varied in its responses to any sort of idea generation. It can give you great ideas one minute, and the next minute it lays out one stinker after another.
To get the most out of it, I have found that there are three main steps to go through.
Train GPT-4 on what you want it to do
Ask GPT-4 to do it
Validate GPT-4’s responses independently (ie. through an additional tool)
Step 3 is incredibly important as GPT-4 is completely delusional on its ability AND on its responses. It will happily proclaim that it has “solved your problem” yet you are left looking at nonsensical garbage, or code riddled with errors.
So now let me show you an example of how I go through these 3 steps to get quick and useful results.
Step 1. Train GPT-4 As a Title-Generating Expert
This step may sound complicated, but it’s extremely easy.
The way I get GPT-4 to “learn” quickly is to point it at a public Headline Analyzer website that is available to everyone:
This site is well-recognized as a tool to validate your article titles.
I can prompt GPT-4 to tell me what criteria is important to create an irresistible title:
Prompt to GPT-4:
Analyze the website provided and give the criteria for creating irresistible headlines: https://www.monsterinsights.com/headline-analyzer
Response from GPT-4:
OK, great, this looks pretty accurate to what is actually displayed on the MonsterInsights website.
Now what this site cannot do is to give you some examples of what a good title might be for your article. This is where GPT-4 comes in - it CAN do this.
Can it do it well? Let’s put GPT-4 to the test and ask it to create some titles based on the listed criteria.
Step 2. Generate a List of Titles
Once I have GPT-4 primed and ready to go, I can feed the article that I have recently finished to GPT-4.
Now for the sake of this tutorial, I am using a recently boosted article of mine:
This is an article that I plugged into GPT-4 to generate the actual title.
I initially wrote the article in a Word document, which I uploaded into the GPT-4 window. The next step is to prompt GPT-4:
And GPT-4’s response:
OK, great! It looks like there are some good ones there — including the one that I actually used.
Now we can plug each of these titles back into the Headline Analyzer to see how GPT-4 does.
Step 3. Plug Each Title Into The Headline Analyzer
This is where we find out how good (or bad) GPT-4 was with creating our titles.
We can take each generated title and plug it back into the Headline Analyzer.
The first title, the one I actually used:
Yes, this title clearly hits the sweet spot - 88 is a pretty darn good score.
Now let’s plug in the rest from the list of 5:
You can see that 4 of the 5 titles generated by GPT-4 score well, and there is one absolute dud.
This is why step 3 is so important.
Without external validation, how do you know if GPT-4 is giving you good titles or if its just blowing smoke up your ass? What if you decided that the title that scored 45 was the best of the lot?
From this list of 5, there are clearly 2 that would do very nicely. I chose the one that was scored slightly higher.
It liked that it was simple and direct — and I like the word superpower.
The bottom line: It is a mistake to take an unreliable Large Language Model (LLM) like GPT-4 as gospel.
You need to verify your results with some sort of an external tool. In this example we can use the external headline analyzer tool to directly validate the results.
I use these three steps now for every article I create.
This is the type of model that works with GPT-4.
And thank you for reading. See you again soon!
If you want to learn more about how to prompt engineer GPT-4, sign up for this free 5-day email course on Prompting GPT-4 for data visuals. Let me know what you think: