Thursday, March 28, 2024

The most important question on a PR checklist is...

Admit it, you thought I  was going to say something about Testing. Didn't you?

While testing is super important and should be part of every PR, the most important question to ask when working on something as part of a team is:


Does this change include anything you need to communicate to the rest of the team?


I've added this to PR templates and have been frustrated when "circumstances" have prevented me from doing so.


Why does this matter?

  • Because if it's something low-level or fundamental, then everyone is likely to need to know it. You shouldn't rely on everyone reviewing the PR or checking everything that gets merged. 
  • Because the change(s) in the PR might break the code other team members are working on.
  • Because a "team" implies working together towards a common goal. Keeping secrets and not sharing things that "teammates" will benefit from knowing hurts the team and the project.


Not sharing important knowledge and information creates frustration, resentment, wasted effort, and more.

Wednesday, March 27, 2024

Doing things different in CI builds

Say you have a large complex solution.

Building everything takes longer than you'd like. Especially when building as part of CI checks, such as for a gated merge.

To make the CI build faster, it can be tempting to have a filtered version of the solution that only builds 'the most important parts' or filters out some things that take the most time (and theoretically change least often).

When such a filter exists, it can be tempting for developers in a team to use the filtered version to do their work and make their changes. If it makes the CI faster, it can make their development build times faster too.


However, there's an unstated trade-off happening here:

Shortening the time to build on (as part of) the CI

creates 

A reliance on individual developers to check that their changes work correctly for the whole solution.


If you get it wrong, developers (mostly?) only work with a portion of the code base, and errors can be overlooked. These errors (including things like being able to build the whole solution) can then exist in the code base for unknown periods of time......



Saturday, March 23, 2024

Prompt engineering and search terms?

The prompts given to LLMs are compared to the text we can enter when searching the web.

Over time, people have learned to write search queries that will help them get better answers.

There's an argument that, over time, people will learn to write better prompts when interacting with LLMs.

However, over time, the way websites expose, format, and provide data that's fed into the search engine so that it will be more likely to be what is shown in the results for specific searches. (A crude description of SEO, I admit.)

What's the equivalent of providing data that will be used to train an LLM?

How do (or will) people make or format data to get the outcomes they want when it becomes part of the training data for the LLM?

Yes, this is like SEO for LLMs. 

LLM-Optimization? LLMO?

GPTO?

Wednesday, March 13, 2024

Tuesday, March 12, 2024

Reverse engineering a workshop

I've been working on writing a technical workshop. I've not done it before, and I couldn't find any good, simple guidelines for creating such a thing.

Having asked a few people who've delivered workshops in the past, the advice I got was very generic and more about the workshops they've proctored rather than how to structure one or put one together.

So, rather than make it up, I started by trying to reverse engineer what good workshops do.

I want the workshop to be fully self-paced and self-guided. If it can be used in group or "instructor-led" scenarios, that'll be good, too, but I don't have any plans (yet) for this.

From looking at many workshops I've completed and thinking back to those I've participated in in the past, I was struck by how many take the approach of showing a completed project and then simply listing the steps to create it. I find this approach to often be disappointing.
Yes, as a participant, I get the satisfaction of having created something but it's not something new or necessarily specific to my needs. More importantly, the reasons for each individual step weren't explained, and the reason for taking an approach when others are available (or even what the other approaches are) wasn't given. This means that I don't get the wider knowledge I likely need to be successful. Is the intention that in completing a workshop, you have the knowledge to go and build other things and the confidence to do so, having done it once before? It probably should be. 

What I find many workshops end up doing (intentionally or otherwise) is providing a series of steps to recreate a piece of software and assuming that's enough for the participants to then go off and successfully create anything else.

Yes, saying, "Anyone can follow our workshop and create X", is great. But that's not the same as a workshop that teaches reusable skills and provides the knowledge needed to go and create your own software.

I want to create a workshop as a way of teaching skills and introducing a different way of thinking about a topic.


Aside: what's the difference between a workshop and a tutorial? I think it's that workshops are longer. Possibly a workshop is made up of a series of tutorials.


After initially struggling, I eventually concluded that a workshop is like teaching anything else. With clear learning goals and a structure, it's a lot easier to plan and create.

In this way, writing the workshop was a lot like writing a book. Only without an editor chasing me for progress ;)

More thoughts on this topic another day. Maybe.
Although, it has got me thinking about what I'll write next...


If you're interested in how my efforts turned out, you can see the results of them here.