❄I tried looking up ice cream puns on the internet, but my browser froze

I’m back in the office playing catch-up. The last quarter of the year has started, and I feel behind on everything. I know I tend to do too much, but is it finally catching up to me? Anyways, let’s catch up on some awesome machine learning!

The Latest Fashion

• Can AI discover new scientific insight? Well, Deepmind just published a paper discovering new efficient ways to multiply matrixes that are larger than 2x2!
• I love the illustrated guide to transformers. Here’s the new illustrated guide to stable diffusion.
• You’ve just learned about stable diffusion? How about using it to generate whole videos. Incredible.

Got this from a friend? Subscribe here!

My Current Obsession

Time. I am currently obsessed with time. October basically just started, and I feel like my entire time is spoken for until November. So so so much to do. It’s mind-boggling. It’s also the first time I struggled writing this newsletter as I don’t have enough time to consume awesome machine learning at the moment. Which honestly is a bummer. Imagine I missed all the awesome stable diffusion developments or how Deepmind casually publishes new ways to multiply matrixes. Time to prioritize heavily, which probably means that I’ll post the bare minimum on social media and Youtube gets to suffer once again…

I have been planning to do another giveaway, this time focused around my Skillshare courses. I will probably give away a 1-year skillshare membership, compute credits for Paperspace GPUs, and some of my favourite books in Python and machine learning. So that’s in the making. I’ll keep you updated!

I’m currently reading up on Stable Diffusion, as I want to create a course for artists on Skillshare. This tool is so cool and fun to work with, I think it’s worth it to share this beyond the realms of machine learning specialists. It’s all in the conception phase (did I mention I don’t have time for even breathing?), but if you have any resources, I’d be very happy to include them!

Thing I Like

When I’m stressed, I tend to get migraines from tension in my shoulders. The only thing I found that helps is massages with a foam roller, and I love it, it’s also nice for recovery after a tough workout, but especially regarding migraines, this is my game-changer.

Hot off the Press

I published the last part of my Euroscipy tutorial on ML reproducibility for science, which talks about Ablation Studies.

I have also revamped the About Me on my website. I know no one actually reads those or would notice a change, but I felt it was time to make it reflect myself a bit more. I have also done a ton of background work, where I reduced the build and deploy time of my website from over 30 minutes to as few as 5 minutes. Very proud of that one. I know I could do even more, but it seems that would be going a bit too deep into the stack. So for now I will just 80/20 it.

Machine Learning Insights

Last week I asked, “Why do we usually normalize the inputs to a neural network?”, and here’s the gist of it:

Data comes in all shapes and forms.

But for this example let’s take a simple case, where we have a spreadsheet of measurements of volcano eruptions. This spreadsheet has all types of information that are relevant for different users of this data, but when we as budding data scientists look at the data, we can see 5 numerical columns right away. We see the height in meters, latitude, longitude, the volcano explosivity index and a measure of maximum seismicity during the eruption.

We have the handy pandas.describe(), and we can see right away, that the volcano explosivity index and the seismicity range somewhere between 0 and 10, whereas latitude and longitude range in their usual value a magnitude higher between -90 to 90 and 0 to 360 respectively, and then there’s the height in meters that maxes out at 6,879m for the volcano Nevados Ojos del Salado in Chile and Argentina.

We know neural networks are basically fancy matrix multiplications, where the input is multiplied by a weight, and then every neuron sums up the incoming results from that multiplication. That sum is what causes problems here.

Suppose we just looked at the results from the first iteration, where we multiply our initial weights that were sampled from a normal distribution, so somewhere in the magnitude of -1 to 1ish. Then the result in the neuron is completely overpowered by any influence the height in meters would have since it has a value up to a couple of 1,000, where the seismicity is somewhere under 10.

During training, we would then have to spend valuable epochs learning weights that somewhat equalise that massively different influence from the get-go. On the other hand, if we normalise all our columns, and the height in meters is now a distribution around 0 with most values falling between -1 and 1, that saves a bunch of computation where a network would hopefully have to figure out that the explosivity and seismicity actually matter as well. Additionally, there is no guarantee that the network ever actually figures that out, so we might wrongly assume that the neural network can’t solve this problem when the scales of our features are just very different.

Data Stories

I can’t resist a good map.

This one came across my feed, and I thought it was just too interesting what website is the most visited in which country. Wikipedia is putting in some serious work here! On the website itself, you can also see local maps to dive deeper into the weird little quirks some countries like to visit. Just super fascinating what the culture is around, for example, newspapers or shopping.

[Source] CC-BY-SA Hostinger

Question of the Week

• Where do you normally obtain data for your analysis?

Post them on Twitter and Tag me. I’d love to see what you come up with. Then I can include them in the next issue!

Tidbits from the Web

• I’m a sucker for “here’s how you grow social media” content. But this one genuinely explains how to add to conversations on the bird app.
• This llama is very majestic.
• I loved this Tiktok about “figuring what I do as a consultant”.