Art forgery, LLMs and why it feels a bit off
24 July 2024 | 12:00 am

For quite a while, I wrestled with an experience that I couldn’t quite put my finger on. If I knew a picture or a text was generated by a large language model (LLM) like ChatGPT or Midjourney, even if it looked good or read smooth, I couldn’t shake this feeling of not caring about it at all.

I’ve always considered myself as someone who’s less interested in how something is made: if it’s good, it’s good, right? (There’s a whole other discussion of the damage these systems are doing to the livelihoods of artists but I won’t go there with this post.)

That was, until I read Austin Kleon’s Show Your Work in which he writes about the phenomenon behind this feeling.

He quotes Paul Gloom’s How Pleasure Works: The New Science of Why We Like What We Like as Gloom, a psychology professor, explains how the pleasure we get from art is not just its colors and shapes and patterns but also what we are told about it.

A group of scientists at the Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB) did a study on this and discovered that

When the participants were informed the following image was a fake, their brain responded negatively before even being shown the piece itself. The person had made a preconceived judgement of the image. A comment made by Professor Martin Kemp of Oxford University says it shows "the way we view art is not rational".

I haven’t fully read the original article (and to be honest, probably would struggle to understand its nuances) but Emilia Sharples referenced it in her article Essentialism and Art Forgery where I learned about it in the first place.

Once I read these, I started to realise that it’s likely this exact same phenomenon in play when I was encountering LLM generated art, images or text. My brain knew it’s not “real” and that the effort put into it wasn’t human effort so it made me less interested in it all together – often even before seeing or reading it in the first place.

If you didn’t spend time writing it, why should I spend time reading it?

I was reminded about this again when Neven Mrgan published How it feels to get an AI email from a friend. In it, Neven shares a story of how they one day received an email from a friend. The friend who sent the email (or the LLM that generated it, who knows) disclosed at the end of it that it was “written by AI” (which sounds much fancier than “generated by LLM”) and that sparked a negative feeling in Neven:

My reaction to this surprised me: I was repelled, as if digital anthrax had poured out of the app. I’m trying to figure out why.

Neven then goes on to discuss a variety of reasons why they think it made them feel like it did. I really like one of the conclusions they come to:

It had simply not occurred to me—and now that it has occurred to me, I definitely do not want small talk and relationships outsourced to server farms. This stuff shouldn’t feel hard or taxing; it’s what our presence here on Earth is mostly made up of. The effort, the clumsiness, and the time invested are where humanity is stored.

In an interview with Swiss newspaper Tages-Anzeiger, Oliver Reichenstein shared similar thoughts when discussing the aesthetics of AI:

It’s disappointing: You know better, yet you still use this AI kitsch. I’m sure you also disappoint many readers: They think that you resort to such tricks because you do not put in enough effort. This might get you a few more clicks. But bad images devalue the work you’ve put into your writing.

and

When I scroll through LinkedIn and an AI image pops up, I just scroll right past it. Kitsch pretends to be what it is not. I don’t want to waste my time trying to give sense to carelessly generated nonsense that never had any.

The future of corporate world

A short while later I saw a post in LinkedIn by someone (I didn’t bother to store a reference to it) who was inviting other people to send their “AI avatars” to take part in a podcast hosted by this person’s own “AI Avatar”. I was stunned and kept thinking, what a silly concept. Maybe I should send my own “AI Avatar” to listen to that podcast so that the creation and consumption would be all automated and all that was left was a pile of resources wasted.

But it’s not just podcasts and emails.

Zoom CEO Eric Yuan wants people to send their “AI assistants” to sit in virtual meetings instead of being there themselves. I get the “meetings suck” mentality that’s prevalent in modern work life but the solution is to stop doing the meetings and finding other ways – like written asynchronous communication – to deal with it rather than sending bots to have those meetings.

A meeting is not valuable on its own. It’s the human connection and the positive outcomes from those connections that meetings should be for.

Yuan says

Sometimes I want to join, so I join. If I do not want to join, I can send a digital twin to join. That’s the future.

That’s not a future I want. I want a future – and to be fair, present day – where if I don’t want to join a meeting, I won’t join a meeting.

Maybe the real reason for Yuan’s desire to have people run bot-infested Zoom calls is because “usage has since [after the height of the COVID-19 pandemic] come down, and Zoom faces a number of business challenges he and I talked about.”

If the goal is really to extend the capabilities of our intelligence, experiences and style, surely the video call would be the first one to go. Once the machine learning systems get better, they can be much more efficient having those discussions in whatever binary language they choose to speak with each other, rather than rendering and recording video.

Oh, there’s also real LLM art forgeries

As I was researching for this and collecting stories and notes, I ran into a great piece by Maggie Appleton about how people sell new kind of art forgeries. Instead of copying an existing painting and claiming it to be real, they generate new artwork using neural networks that have been trained by the art of real artists and asked to generate new pieces in their style.

But I'd never seen this collection of his work. I was shocked and delighted to discover [William] Morris was somehow also a fine-art illustrator?? No one had ever told me. And a way better illustrator than I would have imagined. These prints echoed his other work, but with far more detail, rich colours, and compositional beauty. Perhaps he did these later in life? Perhaps the other work I'd seen was by a younger, less skilled Morris?

Instead of disclosing that’s what is happening, these pieces are being sold as original art by the famous artists.

I rechecked the Etsy print listings for disclaimers or details I must have missed the first time around, but none appeared. Nothing said these prints were “inspired by” or “influnced by” or “made to mimic” or any other derivatives of Morris, rather than originals.

A wonderful but scary conclusion to Appleton’s post is the realisation that while now it’s rather trivial to check if these are actual artwork by these artists by doing reverse image searches and looking for their actual art online, the next models will be trained with these forgeries in the training data.


Notes are a tool for …
22 July 2024 | 12:00 am

… thinking

Writing notes is less about the content of those notes being available later. The process of writing them is a process of thinking. It’s a tool to go through information and ideas, and transform them from abstract to tangible.

In Writing is thinking, Francesco Puppo writes:

Putting your words on paper, even if digitally, forces you to deal with the core of the topic at hand. Writing forces you to think about how you express your thoughts. Your words need to stand alone on the paper. No one can read your mind so you need to be clear. And to be clear you need to fully understand the topic at hand.

While he makes a distinction between note-taking and writing in his article, I do think for this part the effects apply to both.

Writing notes and journaling provides me mental clarity and focus.

Active writing and reviewing my notes helps me make new discoveries as well. Without my notes, each thought would risk staying alone as separate thought, disconnected. With the notes, I can accumulate understanding and a knowledge base to learn more about the topics I’m interested in.

I often write notes with pen and paper on my various notebooks because it feels easier for me to connect with my thoughts compared to writing on a computer. When I then bring in those hand-written notes into my digital system (for easier sync, search and access), I don’t just copy them word for word but rather reprocess them. I add links between my notes, add references and resources from other places and deepen my thinking.

… writing

Writing in this blog and in a newsletter, having great notes makes writing process easier and smoother and it improves the quality of the finished writing. I’m not reliant on only what goes through my mind at the time of writing but can dig into my notes and let them help me add ideas that have been thought through more.

The existence of a note of a topic I might write about in the future makes me more likely to notice more articles and social media discussions and books on the topic. And as I have a note about them, it’s easy for me to gather information and references into one place where they are easy to find at time of writing.

It also helps me find more interesting things to write about as the ideas marinate in the notes for a longer period of time.

Christian writes about this in Preparing Fragments Helps You to Ease Into Writing:

A Zettelkasten makes writing texts easy. It encourages you to prepare research and the most of your writing before you compile your first draft. This way you can focus on one task at a time and needn’t sweat about getting through. This works excruciatingly well with longer texts but it’s proven indispensable for any of my shorter writing projects, too.

This is something that people who use the Zettelkasten system often talk about: notes being a sparring partner and a very useful tool in writing. I’ve taken some ideas from Zettelkasten but don’t really use myself but I do find the benefits with my note-taking system as well.

Christian also writes:

You should do research and writing separately because it’s too costly if you don’t. To get into a writing flow takes time. It’s cost-inefficient to constantly interrupt your writing just because you find your research foundation to be lacking. Doing research requires another mode of attention which, too, takes time to get into. Switching between these procedures or a regular basis is an uncomfortable waste of energy and, ultimately, a waste of your precious time.

I still write a lot of my blog posts on a single sitting, with no or very little research when they are mostly musings of the daily life but whenever I want to write something more in-depth, I turn to my notes to help guide me through the act of writing.

… learning

Active learning through making notes improves understanding and recall. Our memories are bit brittle so notes are not only great at storing the information in a way that can be pulled up when needed but they also serve as a bouncing board for ideas.

When I was a kid in school, I wrote what I considered back then extensive notes. But what I was mostly doing was just writing down everything the teacher said or had written down in their slides or blackboards. It helped me get through school but looking back now, I would approach those notes differently and write more reflection and rephrasing of the study content.

Active learning and critical thinking help with retention. There is a measurable difference between passive and active learning in terms of how much information students are able to retain. Our ability to recall information is reinforced when we employ higher order learning techniques such as applying, explaining, evaluating and creating. - British Columbia Institute of Technology

What started to make a big difference in my notes was when I started to make connections between them, referencing other notes and actively going through them whenever I was processing new information and ideas.

The best part is when my notes system surfaces old notes that I didn’t remember making. Every time I write a new note and browse through connected ones, I go through a short learning enforcement journey.

And like writing for public, when I write my notes, I do a bit of extra work to find references and links so I can refer to them in the future more confidently rather than just trusting my own thoughts or memory.

… productivity

Good notes help me be more productive as well. I can offload tasks to task lists, track my progress and record history of my actions like taking meeting notes at work or recording milestones for projects.

Just like processing my thoughts into notes brings clarity of mind, same happens with these productive notes. When I know I don’t need to worry about forgetting to do something or not remembering what was agreed in a meeting, I gain clarity that helps me focus on the work at hand.

At work, it also helps me with my reviews and career progression as I keep track of what I’m doing as well as maintain a brag doc. It’s easier to maintain one when I can refer to different notes detailing the circumstances and work done. As Julia Evans explains it:

The tactic is pretty simple! Instead of trying to remember everything you did with your brain, maintain a “brag document” that lists everything so you can refer to it when you get to performance review season! This is a pretty common tactic – when I started doing this I mentioned it to more experienced people and they were like “oh yeah, I’ve been doing that for a long time, it really helps”.


Home-cooked, situated software
20 July 2024 | 12:00 am

For July 2024, James is hosting the IndieWeb Carnival and he picked the theme of Tools for us to write up:

Over the next month, I invite you to write a blog post about tools and how they do, or have, influenced your createive process. You can write about tools in the context of any creative practice you have: writing, acting, drawing, painting, woodworking, cooking, coding, singing, VTubing, or whatever is applicable to you.

I’m extending James’ invitation to you! If you have a blog, why not join the carnival and join our jolly crew. We have a really fun community.

There are so many things I could write about. I have earlier written about how at the beginning specific tools matter less but as your skills improve, they become very important to master so I won’t go there today. I have written about how blogging is a great tool for refactoring and I have shared some of my favourite tiny software tools and tools and services that I use daily.

I’m a Saturday evening hobbyist

As a software developer, I mainly consider myself as a “Saturday evening hobbyist”: I love software development but the part of it that I enjoy the most is build small things for myself, my family, my friends and my communities rather than doing full-time professional software development (which I occasionally do to pay the bills).

Explaining it felt bit cumbersome though because it wasn’t easily clear to other people what that meant. That was until I read the notes and later saw the recording for Maggie Appleton’s Home-cooked Software and Barefoot Programmers (blog format) which lead me to learn how different people have described my approach in different ways. In her talk, she talks about how large language models (LLMs) can have a revolutionary impact on how non-professional developers can build software for their need.

I won’t be talking about LLMs but explore the ideas she shared about home-cooked software and how it relates to my relationship with software development and building tools.

Home-cooked software

The term of home-cooked software comes from Robin Sloan’s An app can be a home-cooked meal from 2020 where Robin describes a mobile app he built that had a small handful of users: his family. It was an application to record and share short video messages to other family members.

In his article, he talks about how he is the programming equivalent of a home cook:

People don’t only learn to cook so they can become chefs. Some do! But many more people learn to cook so they can eat better, or more affordably. Because they want to carry on a tradition. Sometimes they learn because they’re bored! Or even because they enjoy spending time with the person who’s teaching them.

He also writes about the freedom that comes from realising that not every software need to be a professional product:

When you liberate programming from the requirement to be professional and scalable, it becomes a different activity altogether, just as cooking at home is really nothing like cooking in a commercial kitchen. I can report to you: not only is this different activity rewarding in almost exactly the same way that cooking for someone you love is rewarding, there’s another feeling, too, specific to this realm. I have struggled to find words for this, but/and I think it might be the crux of the whole thing: [\n] This messaging app I built for, and with, my family, it won’t change unless we want it to change. There will be no sudden redesign, no flood of ads, no pivot to chase a userbase inscrutable to us. It might go away at some point, but that will be our decision. What is this feeling? Independence? Security? Sovereignty?

Situated software

Both Robin and Maggie link to Clay Shirky’s article Situated Software from 2004 where Shirky, who taught Interactive Telecommunications Program in NYU, talks about a paradigm shift he noticed in students’ approaches to building software services.

The biggest difference this creates relative to classic web applications is that it becomes easy to build applications to be used by dozens of users, an absurd target population in current design practice. Making form-fit software for a small group of users has typically been the province of banks and research labs -- because of the costs involved, Web School applications have concentrated on getting large-scale audiences. And by privileging the value that comes with scale, Web School applications put other kinds of value, particularly social value, out of reach.

It felt so reassuring or relieving to find other people talk about what I’ve been feeling and giving it better names, terms and analogies than I ever could.

I really like the first examples Clay shares:

The first had to do with reputation systems. One project, The Orderer (designed by Vena Chitturi, Fa-yi Chou, Rachel Fishman, and Cindy Yang) was for coordinating group restaurant orders, common in late-night work sessions. The other, WeBe (Brandon Brown, Yoonjung Kim, Olivier Massot, Megan Phalines) was a tool for coordinating group purchases of things like chips or motors. Because money was involved, a Web School approach would require some way of dealing with the threat of non-payment, using things like pre-pay or escrow accounts, or formal reputation systems.

I often hang out with professional developers: I run meetup communities, chat online with developers, attend conferences, and read a lot of blog posts and guides by fellow developers. A big chunk of the technical discussions there are about professional software and “best practices” for them.

But I love building what Shirky calls situated software. Software that doesn’t have to be universal, general and productised.

The gap between “just enough for us” and “public production-grade” software is massive and honestly, very little of that extra work is something that excites me. To bridge that gap, one needs to do a lot of work. There’s work to be done in infrastructure: scaling servers, authentication and user management (with all of the signup/recovery flow that it comes with), security and so on. And the user experience needs to be much more polished when the product is aimed for a larger audience.

One of the fun parts is also that when building these software, we don’t need to worry so much about “best practices” (I put them on quotes ‘cause I’m not convinced there are any). Rather: if it works, it’s good to go. The surface area for bugs and security problems is so significantly reduced compared to professional software that it’s okay to not follow all the patterns they teach in software engineering.

Some examples of my home cooked software

Party selfie app

One of my favourite examples of these types of trade-offs is from a small software I built for one event a few years ago. The company I was working for was sponsoring a social event and we wanted to have something fun for people to do while having a great time.

The application worked like this: there was a list of categories for which you had to find someone in the event who fit that and take a selfie with them. Think of categories like “a cat person” or “alumni of class 2015”. They would then post these images to a Telegram bot that I wrote, which would send them to a local Flask server that stored them in a local sqlite database and a simple frontend that showed the photos in real-time on a screen at the event.

To make this into a proper, production-grade application, there’s a lot one needs to do to build polished UX and add security and authentication features. Since it was meant to be used only once in that event and I had other work to do as well, I didn’t want to spend all that time to work on the extra stuff.

That’s why I kept everything local: to get started, someone needed to start a local dev server and open the localhost address on the browser to show the photos. The images were stored on the laptop and not shared in the web and everything could be easily deleted with one command at the end of the night.

235

Another example is 235 which is a command-line tool I built for myself to get NHL results into my terminal. It is a bit more polished and it is publicly available for anyone to install but it’s mainly built for myself, providing the features I want it to have. It was originally built as a project to learn Rust.

Pokemon TCG tools

Gym Leader Challenge Decklist Validator is another one. It’s purpose-built for our community of Pokemon TCG players and has a simple design. It has a small handful of users (to my knowledge, I have no analytics) since it’s a very niche use case.

Even more niche is a tool that I’ve been building for myself to build Pokemon TCG decks. As the official Pokemon TCG Live’s deck builder is not to my taste, I started building my own. It’s not finished, it’s not published but I use it weekly by starting a local dev server for my React project, use it and when I’m done, I close the dev server and go on with my day.

Tooling to run this site

IndieWeb wiki lists the key principles of the community, two of them being:

💪 Make what you need. Make tools, templates, etc. for yourself first, not for all of your friends or ”everyone“. If you design for some hypothetical user, they may not actually exist; if you make for yourself, you actually do exist. Make something that satisfies your needs (also known as scratch your own itch), and is compatible for others, e.g. by practicing POSSE, you benefit immediately, while staying connected to friends, without having to convince anyone. If and when others join the indieweb, you all benefit.

and

😋 Use what you make! Whatever you build you should actively use. If you aren't depending on it, why should anybody else? We also use the metaphor eat what you cook for this principle. Personal use helps you see both problems and areas for improvement more quickly, and thus focus your efforts on building the indieweb around actual needs and consistently solving immediate real world problems, instead of spending lots of time solving what may be theoretical problems.

My approach with building this website and the tools made to build and run it matches these principles. This is also what I’ve been telling students and juniors who want to get better at software development: build something that you need and will use because it makes it easier to find improvement ideas and be more motivated.

The tooling for this website and blog is a collection of custom scripts and software, mostly run either as Javascript functions with Eleventy during build time or as command-line tools to fetch blog posts, to manage post relation data, to handle Mastodon comments, to update my “Most popular” posts list and so on and so on. These tools are very specific to my site and they are not something that anyone else could pick up and add to their project as-is.

Ad-hoc one-off scripts

Sometimes these tools don’t even reach the “tool” stage. I weekly write short scripts of Javascript that I run in the console to do things like extract Youtube playlist videos into a Markdown bullet point format. These are tiny tiny scripts that are almost always one-use with specific use cases so I rarely store them in any files because it’s often easier for me to build them from scratch every time than to try to modify existing to a changed environment.

These can look something like this:

let talks = $$('#contents ytd-playlist-video-renderer') // Using Firefox's shortcut $$

Object.values(talks).map(talk => {
	let title = talk.querySelector('#video-title').textContent.trim();
  let href = talk.querySelector('#video-title').href

  let url = new URL(href);
  let videoId = new URLSearchParams(url.search).get('v')

  return `[${title}](${url.origin}${url.pathname}?v=${videoId})`
}).join('\n')

Another tiny local script that I developed during the writing process for this blog post is my shell utility ,table which allows me to generate a Markdown table template with any amount of columns to speed up the creation process.

All of these examples are the software equivalent of home meals that I cook to keep myself going, not for their monetary value or to be served on a restaurant menu.

I’m happy I can do this

Software development is a quite nice profession to have: the salaries are great, the work culture is often good (thanks to the big demand for devs) and you get to build tools and services used by people to solve their problems.

That’s not why I love it though. I love it because my ability to build what I need for myself, my friends & family and my communities enables me to make things that truly matter. These days, I don’t even notice every time I do it.

Even though my apps are not always usable as-is by other people, I aim to publish near everything as open source so if someone wants to see how they are built, they can. I also share my learnings and snippets and ideas as blog posts to inspire others.

Only once I started drafting this blog post, did I realise how much I rely on tools built by myself on a daily basis (and enjoy using them).

I wish everyone would have someone in their life who could help them with custom software. In Finnish, we often call these people “kotinörtti” (”house geek”, a play on the Finnish House Elf folklore) and they are often family members who help fix printers and solve other computer problems.

At the same time, I hope that as an industry, we’d also learn to not see ourselves as the universal only tool to solve problems – especially when those problems are not technical.

You can build own tools without coding too

Programming isn’t the only way to build tools either. It’s maybe the most efficient and powerful since there are few limitations but you can build all sorts of fun stuff with office tools like Powerpoint/Keynote/Google Slides with a bit of tinkering and imagination.

Youtube channel ZaneGames uses slides to create all sorts of fun Pokemon quiz games to play with his friends on their channel. Emeral went to the distance and built a VR Beat Saber style game with Powerpoint.

Back in the day, I taught idea phase startup entrepreneurs how to build mobile app prototypes with Keynote to communicate their idea.



More News from this Feed See Full Web Site