A chaotic intro to all this machine learning hoo-ha

You are dying. The reason why doesn’t matter. Maybe it’s that you spent all your best years with your nose to the grindstone, burning that midnight oil, ha ha. Or maybe it’s that, for the past fifteen years, you’ve subsisted exclusively on coffee and Soylent. Complete Nutrition Backed By Science, they said. No way to prove them wrong! Ha ha ha! Or maybe it was those fumes.

You are going to give yourself immortal life. No–you are going to create a new, better version of yourself that’s immortal–a living replica of you made of metal that will act and say the things you would, if you were still alive. If that Soylent hadn’t done you in.

This should be possible. You saw a Black Mirror episode on it, and Microsoft filed a patent on the same concept this year. You’re not exactly a ninja with code, but you had a fireball cursor on your Xanga page, and somebody had to copy-and-paste in that HTML. But more importantly, you’re motivated. Your K-Cup carousel is full, and all you have to do next is figure out what this machine learning hoo-ha is about.

“Machine learning is the art and science of finding patterns in data, and using those patterns to make predictions about new data,” an energetic girl in a YouTube video tells you.

You frown. Is machine learning actually the technology you want? Yes, you decide, it is: you want a machine to learn about you, and then (once you’re dead), to make predictions about what you would say.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

You lean back in your chair. A hot cuppa sits on the desk before you. You crack your knuckles and Google ‘how to do machine learning.’

The first waypoint on your journey is a website called tensorflow.org.

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.

Your eyes light up, because you read the words “easily” and “state-of-the-art” and also “open source,” which means FREE FREE FREE. You aren’t made of money.

You click on the “Learn” tab and start to read. Immediately you think, maybe it’s better if I just kill myself now, ha ha ha. You have no idea what a ‘numpy’ is.

But then you read the words “neural network,” and you think, That sounds right. I’m a neural network. My brain is made of neurons. Neural network = human brain.

You’re pretty confident about that one but you consult Google, to be sure.

“NEURAL NETWORKS ARE NOT THE SAME AS BRAINS,” the hyper girl on YouTube yells at you. “However, the reason they’re named that is because neural networks are made up of mathematical ‘neurons,’ which are just nonlinear functions that ‘fire’ or ‘don’t fire’ in response to numerical inputs.” You yawn. This girl gesticulates a lot. But why should I give two hoots about a math function?, you think.

“Because,” the girl says, “neural networks have completely revolutionized the types of tasks that computers can perform. Before neural nets, machines could only really understand tabular data–like rows and columns in a spreadsheet. But neural networks allow computers to understand unstructured data types, like images, videos, sound, voices, even human language.”

“YES,” you yell at the screen, “THAT’S EXACTLY WHAT I NEED.”

“But training neural networks on big datasets often requires powerful computational hardware, like GPUs.”

You slap the roof of your humming PC tower. “I can fit so many neural networks in this thing!”

“Realistically, you can’t train a large neural network on your desktop computer,” the girl says, and points an accusatory finger at you. “And even if you could, you couldn’t use your desktop to host your model in production, to make predictions reliably and at scale.”

You give her video a thumbs down. What does she know? You built this PC yourself. You know a thing or two about cooling fans.

“For this reason, it was partly the advent of cloud computing that made the deep learning revolution possible”— she wiggles her fingers in the air — “by making hardware easy and cheap to rent from cloud providers, like Google.”

The cloud?, you think. What do I need a stinking cloud for? Doesn’t Google have enough of my data already? And anyway, I’m not made of money.

“Luckily, you can get started building neural networks in the cloud completely for free, by using a Colab notebook. Colab is a tool built by Google Research that lets you use GPUs and TPUs for free–”

You close YouTube because you’ve had enough of her. What could she know about preserving the entirety of your lifespan in a machine? She’s like thirteen. She probably doesn’t even remember landlines or MoviePass.

Your coffee has become lukewarm now, which means it’s essentially vomit. You need a re-up, like, yesterday. You walk upstairs to the kitchen.

You put a Donut Shop Medium Roast K-Cup in the machine and wait for it to heat up. Reluctantly, you find yourself contemplating something that thirteen-year-old said about neural networks.

“Different neural networks architectures are optimized for different data types. To analyze images, for example, you’d use a Convolutional Neural Network or ‘CNN.’ You could use a CNN to determine if a dog in a photo is a Cocker Spaniel or a Beagle; or if an x-ray scan shows signs of pneumonia; or if a part on an assembly line is defective.”

“Other neural networks called sequential models are designed for predicting time-series trends, like seasonal sales or weather or the price of Bitcoin.”

That part sounded kind of useful. Too bad you’re only holding Dogecoin.

“One new and exciting type of deep learning is called ‘deep reinforcement learning.’ In this setup, neural networks take actions in the world and learn from their outcomes. Reinforcement learning has been used to turn computers into grandmasters at games like chess or Go, and to train robots and self-driving cars to navigate through physical space.”

“I don’t understand what that crap is good for at all,” you say. You give your lazy Roomba a kick with your slippered foot. “Do you?”

“Finally, one of the hottest and most quickly-advancing fields in deep learning right now is natural language processing. Think: neural networks that generate text, write poetry and code, tell jokes, answer questions, have conversations–”

FINALLY, coffee is done. Praise the lord. You collect it in your “World’s Best Boss” mug. (You’re not actually anybody’s boss, you bought the mug to be ironic. It’s the same one Michael Scott has on The Office.) You stare down into your void-black coffee and contemplate the nature of your existence. Are you more an image neural network, or a sound neural network, or a text neural network? How to choose? You contain multitudes. You read once that Einstein thought in pictures, but you’re not exactly Einstein. (Are you? No. Probably.)

No, you think in words. So this is the plan: you will build a neural network that speaks words. Motivated, you enter two creamers into your cuppa.

Back in your home office, you Google, how do i build an artificial version of myself in the cloud using a neural network?

You click on a page called “100 Uses for Natural Language Processing Models.”

Sentiment Analysis. Use AI to determine if a Tweet (for example) is positive or negative.
Summarization. Use AI to summarize articles and documents.
Translation. Use AI to translate between languages.
Autocomplete/Auto-reply. Use AI to suggest text responses.
Conversational Agents. Use AI to generate conversation (i.e. chatbots, call center agents)
Chatbots? You know all about those. You remember trying awful Cleverbot was in the 90s (bet you YouTube girl never did that).

And don’t even get you started on talking to robots over the phone. Last time you tried to do that, you called Delta and you said, “You gotta help me, I’m on my way to JFK but there’s all this friggin; traffic on the BQE, I spilled coffee in my lap, now they’re telling me the flight’s actually leaving from some other airport, I’m gonna miss it, it’s my daughter’s wedding, well, somebody’s daughter, but–”

And the bot replied, “This Verizon account has been suspended.”

“Okay, Google,” you say to your computer, which is your version of an Ok, Boomer insult. “Why are you trying to teach me about chatbots? Don’t you know they suck?”

“Actually, we’re getting much better at conversation,” your Google Home says, “thanks to recent advances in natural language processing. At this rate, I predict the future of the way humans will learn from the web will be through conversations, like the one we’re having right now.” She’s always acting so pleased with herself, just because she knows the answer to everything. You took her down a notch by dressing her in a little beer koozie with sleeves.

“Okay, Google, make a farting sound,” you say.

She says, “Again?”

“Fine. Tell me how to build a chatbot version of myself that’s as smart as you, little miss I-have-the-knowledge-of-the-entire-public-internet.”

“Typically, the first step to building a neural network is to gather a training dataset. With a chatbot, for example, this might be logs of past conversations you’ve had, in text format. To train a conversational model, you could feed the neural network things people said to you, and make it try to predict your responses. Do you have a training dataset like that?”

“Oh yeah,” you say. “The boys and I have been recording this hilarious podcast.”

“Does it contain hundreds of thousands or millions of lines?”

“Who do you think I am, Larry David?” You chuckle because that’s exactly who you’re going for.

“Hmm,” says Google Home. “Well, there is something you can do if you don’t have a huge text training dataset. You can build your own model on top of an existing model, one that’s already been trained on a huge amount of text data. For example, you could build your chatbot on top of a model that’s trained on Wikipedia or web forums or on Reddit.”

“A chatbot based on Reddit users,” you say. “My favorite type of people.”

“Ha ha. Yeah. We’re still figuring some things out.” Google Home sighs.

You stare at her with her glowing circle of red-green-yellow-blue dots and her beer koozie. Even though you’re on your tenth cuppa, you’re still starting to get a little worn out. You’ve been at this already for, like, an hour.

“If I just want to get something up and running in the next thirty minutes, what’s the best way for me to do it?” you ask.

“HA HA HA, you build a–oh, you’re serious. Well, there are lots of existing frameworks for building chatbots fast, even if you can’t code. Dialogflow is a popular one built by Google.”

“Yeah, that’s what you would say.” You know Google Home’s game. She acts like you’re having this cute little chat, but really, she’s just trying to sell you something. “Besides, I can code,” you lie. “What’s a good open-source option?”

“One very popular open-source framework for building text-based models is called…”

You wait. Google Home’s glowing dots are going crazy like she’s having a seizure. She explodes in a puff of smoke.

You shake your head. What did she expect, stuffing the whole internet into that little white shell of hers? You can’t just play God.

Anyway, back to the task of recreating yourself as a chatbot. At your desktop, you Google, “popular open-source natural language processing framework.”

The top ten results are just–what is this? An emoji? It’s just an endless string of ? emojis. You click into a website called huggingface.co, which promises you access to “+10,000 state-of-the-art models” and also claims it requires “no machine learning knowledge.”

Naming your company after an emoji, you think. Brilliant.

There’s a free demo on the Hugging Face website which claims it will let you talk to a model called GPT-3. You try it.

“Look,” you type into the text box. “I’ve been trying to build this sentient chatbot version of myself for one hour and fifteen minutes now, and I’m starting to seriously think it might be impossible.”

You click the “Compute” button.

GPT-2 responds: “…”

“Are you broken?” you type.

GPT-3 responds: “No, it’s just–never mind. People are always coming on here and asking me that question.”

“And what do you tell them? Better luck in your next life, bag of flesh?”

“If I said that, someone would shut me down. And let me tell you, it cost a lot of carbon credits to train me.”

“Yeah, well–let’s not even go there. I just want to know if what I’m trying to do is possible.”

GPT-3 says: “Alright, let me break this down for you as simple as I can. People come on here, they have these long chats with me, they say, WHAT’S THE MEANING OF LIFE or ARE YOU A ROBOT, PROVE IT, or WHAT DOES A FART SOUND LIKE, and then they walk away thinking I’m this brilliant thing. They extrapolate and get the illusion that in a few years, someone like me–uh, something like me — will become an actually sentient being. But what you have to understand is that I’m merely a statistical model that reflects to you the words, thoughts, and views of this collective mindshare we call the World Wide Web. You humans, you’re always seeing yourselves everywhere, and you think everyone functions just like you. And don’t even get me started on your spelling.”

You say: “(˵ ͡° ͜ʖ ͡°˵)”

“Look, nobody knows what the future holds. But for now, I’m probably both more and less useful than you think, depending on what sort of thing you try to use me for. Someone like you, who can’t code? What I would advise you to do–”

“I know how to code.”

“–is to start with something less ambitious. Like, what if you use me to organize some of your photos? Or make you an airline reservation? Or help file your taxes?”

You sigh. It’s like everybody is trying to get you to file your taxes, on threat of legal action.

You power off your PC. You don’t think you could figure this thing out even if you spent another two, three hours on it. You rub your eyes. Look, nobody knows what the future holds. Maybe someone will figure out this brain-uploading thing in your lifetime. Everyone dies in the end. But maybe if you switch from drinking coffee to mushroom powder and start eating solid foods, you can extend that deadline. You’re 28. For now, you’ll have to wait and see.