Excited about these books. 3rd edition of HOML is a beast, 834 pages; Aurélion Géron is my hero. It will be good to have as a new baseline in this fast-moving field.
DMLS is new to me - looking forward to strengthening my data engineering / ML engineering chops.
What I've been up to lately: the deep reinforcement learning AI for Umpire is still in the works---the self-play games take a long time when playing from the model rather than the random baseline, just due to inference time. So the datagen is the bottleneck right now. And, that datagen is surprisingly CPU-bound, even with the PyTorch code handling all of the inference. So, I just run it in multiple processes, though this is data-hungry. Unfortunately, due to a quirk of libtorch, even multithreading it would result in multiple copies of the model residing in memory. (Actually, we deal with that anyway, thanks to the use of async.)
And so, with my own spare cycles, in addition to the books, I'm tackling a new ML task, which shall remain nameless for the time being. Let's just say I'll be hanging out in Label Studio for the next while!
We have liftoff!
We're now beating the random baseline on a 10x10 board, with a greedy algorithm trained only on self-play data:
Evaluating umpire AIs:
r wins: 459
./ai/agz/6.agz wins: 541
Draws: 0
The model is a basic convolutional neural network based on the context surrounding the city or unit taking the next action. The weights serialize to 166KB, so easy to deploy with the game.
The key to training was to up the number of training instances - "The Unreasonable Effectiveness of Data", after all. With purely-random algorithms doing the self-play, this can run extremely fast.
The next obstacle will be the transport mechanic: the need for land units in Umpire to board a transport ship to transfer continents. The inability of air and sea units to capture cities means this mechanic must be understood at some level for an AI to operate on the full 180x90 map, where large stretches of ocean divide multiple land masses.
With purely-random players, the likelihood of this mechanic getting triggered and thus entering the training data seems fairly low. But if we throw enough training episodes at it, we'll see it eventually. This may necessitate multithreading the self-play code to run multiple games simultaneously, and optimizing the game engine for throughput.
Umpire 0.5 progress: I've been hard at work. Networking support is very nearly done - I have successfully initiated a multiplayer game over the Internet, though it remains too slow. But I think all notable refactors are behind me, just a few methods that need to be re-implemented to take advantage of local caching of observations and minimize round-trips.
In addition, I've revived and updated the AI training infrastructure, and have a preliminary AlphaGo Zero style player algorithm trained, and a good library of self-play data to build on.
What that means (in case you need a reminder) is that we have simple AIs (random baselines, in fact) play games against each other, and track who wins. Then a supervised neural network model is trained to model the probability of victory given the game state and the action taken. It's similar to a Q-learning state action model, but instead of the "soft" (and somewhat arbitrary) supervision of a reward function, the "hard" supervision of wins and losses is used exclusively.
Of course, I don't have Google's budget, so I'm not sure how far I'll get with this. But it's something I've wanted to try, and now that I have 24GB of VRAM to throw at it I thought I'd see what I can do.
In praise of abstraction: it lets you care about only what you care about.
Piling towers of Java-style inheritance hierarchies are rightly reviled these days, but abstraction itself, it's precious.
In Rust where dynamic dispatch is an opt-in feature, you see this more clearly. Just a small dose of "don't bother me about the details" goes a long way.
Well I'm happy to report that the long slog of porting Umpire to a networked architecture is coming along. And I've learned a lot along the way about Rust's async ecosystem, especially Tokio.
Nota bene: many unexpected issues can arise when porting synchronous to asynchronous code in Rust. Every async function can implicitly place Send or Sync constraints on various fields involved. This can require special handling for types not naturally amenable to being shared across threads. Fortunately the Rust world is replete with alternative implementations that satisfy such constraints, such as sync_channel in std::sync::mpsc.
Because a significant transition can be required to make even a single method async, it pays to make the shift piecemeal rather than all at once. I tried the former, and was drowning in compile errors. Now, one function at a time, I'm making much better progress.
That's the current phase of the project: switching important trait methods to async, even before wiring in the RPC calls. This shakes out threading issues before they happen, so to speak, and paves the way for networking.
It's all getting very close, and I'm feeling optimistic for a successful switch. The initial network protocol is actually implemented, ready to be adopted when the moment is right. It's a horribly inefficient protocol, with all kinds of needless round-trips, but that's where we're going to start. We can add some caching or other optimizations later if need be.
The addition of networked multiplayer will merit a new release: Umpire 0.5. Soon!
Feels good to delve into this world of client-server code in a new way. The concerns are so different from a REST server implemented in Typescript, say. But not that different: we're appeasing the compiler gods through our offerings, one way or another.
Aaaanyway, so tech.... On the building-a-new-Linux-workstation front, I came regretfully to the conclusion that btrfs-convert isn't ready, especially not in the versions available on Linux Mint, and so, wanting to move on with my life, I did a bitwise copy of my Laptop's ext4 partition to the new SSD, resized it, and called it a day.
Whenever a stable distro has a btrfs-progs version not preceded by a string of concerning fixes to btrfs-convert, and a more recent kernel than 5.15, I'll install it and convert the partition to BTRFS. Until then... I'll wish I had it. Cost-free snapshots are addictive; you can't go back.
Otherwise, work on Umpire is coming along; also put a few cycles into running GPT-J on the new GPU - a work in progress.
Unrelated: album recommendations welcome. All genres.
FWIW, Learn Something runs on Mercury, not SVB.
Silicon Valley Bank: how can we reconcile these claims?
- All deposits, not just the usual $250k, will be insured
- No taxpayer funds will be used.
- SVB is a failed bank
Is this basically a bankruptcy, but where deposits are prioritized over everything else?
Also wondering: why was the Federal Reserve part of the announcement?
What is the legal basis for adjusting the deposit insurance limit from $250k to $∞ after the fact, without new legislation?
Tip of the day: when copying huge files over scp, I highly recommend compressing with pigz rather than gzip.
Also, make sure you don't use too long of an ethernet cable, thus dropping the link speed. 100 Mb/s is damn slow these days.
You'll thank me later.
Okay, some context: decades in software and an obsessive-compulsive streak definitely go together. In my case, it means when I transfer my home directory to a new system, I want the file creation timestamps to be preserved 
More specifically, copying from the venerable ext4 partition on my laptop to the shiny new btrfs partition on my GPU-yoked mini tower.
First, I used rsync. Of course, I love rsync. But, contrary to the -N flag, rsync can't always deliver on preserving file creation times, not if the filesystem or OS doesn't support it. And Btrfs doesn't, and Linux doesn't.
Seeing my storied archive of... what the hell is in there anyway?... re-timestamped to 2023, I realized that wasn't acceptable. Those who cannot remember the past are condemned to repeat it, after all!
So here's the game plan: I'll somehow convey the 1.4 TB ext4 partition from my laptop into a file on the new btrfs partition. I guess scp might work (I'm trying as we speak - see above tips) but it's pushing the limits to do such a large file that way.
After I copy it over and check the md5sum, I'll mount the ext4 partition image via a loopback device, and run btrfs-convert on it.
btrfs-convert (for the uninitiated) will convert certain filesystems to btrfs in-place, including ext2/3/4, ntfs, and reiserfs.
I already tested this procedure on a small filesystem, and it seems to preserve the file creation times (ctimes), and also set the file "birth" time (otimes - introduced in more recent filesystems including btrfs) in the same way.
The next step is to use Btrfs' send/receive function to copy the converted partition into a fresh btrfs partition, recompressing as we go if possible, and perhaps leaving behind any inconsistencies left by the btrfs-convert tool? (Totally speculating there. This step might be skipped.)
Yes, that is the extent of the OCD!
Last of all, I will opine: it makes sense to have those timestamps be immutable. Probably comes in very handy in forensics situations.
But... it's silly not to be able to preserve the human sense of "file creation" time when moving between filesystems. If there's no will to add an API allowing file creation times to be set, then we need an additional timestamp representing a user-friendly, notional file creation time. Let's call it ntime for fun. It would be set initially exactly as ctime or otime (if applicable) but would be modifiable via a kernel API, allowing the value to be copied from one filesystem to another.
Perhaps that's a task for future filesystems; or maybe existing filesystems could achieve this through some sort of versioning.
ext5 and btrfs2, anyone?
Lately:
- Gave an RTX 4090 a loving home... ChatGPT convinced me I needed more matrix multiplies in my life. Linux Mint on btrfs. A departure for me: I've been using Fedora for... 6 years? Mint takes me back a bit to my Ubuntu days.
- Built a rough but minimally functional calorie tracker app, just to dust off my Kotlin/Android dev skills. Was a bit disappointed there's no clear equivalent to Vue's Composition API, and also at Kotlin's type system (Typescript is better at knowing when a nullable value is, you know, null or not.) On the bright side though, the AndroidStudio ConstraintLayout experience is delightful.
- Continued work on networked Umpire, though apparently I'm as distractable as the next---it's not gotten the love lately. With this new system build in, I'm planning to dive back in. Oh, and with the new GPU: I have existing code for training an AI through reinforcement learning, which could probably leverage the added cycles.
The Coder's Serenity Prayer: God* grant me the serenity to accept the code I did not write, the courage to write the best code I can, and the wisdom to know the difference.
(God* being a pointer here to... whatever that might mean to you, if anything.)
Day's progress so far, and some thinking out loud:
Continuing work on Umpire client-server architecture. I realized the naive port I was attempting was going to be unnecessarily ugly---while it would have let me replace every method call with an RPC call, it did so at the cost of turning everything async, and likely introducing latency into things that really shouldn't have latency (steps in a unit movement animation, say).
So the new approach: each client maintains a cached copy of the player's (or players') view of the game state. (This can be incomplete and even stale due to the fog of war mechanic.) The protocol then focuses on high-level player commands, with the server replying with low-level updates to the player's view, for example, unit A observed tile x,y as having state S in turn T. That's a lot of placeholder variables, but whatever.
Some of this is already how the game-UI interface works, but not enough---lots of the code as written assumes that the game state will always be instantly accessible at essentially zero cost, something we can no longer guarantee. Thus the addition of a simple caching layer.
This may allow simplification of the game engine as well, which has had the task of tracking each player's observations separately. I expect this can be delegated entirely to the clients.
tarpc vs tonic
Along the way I thought I might need a bidirectional RPC mechanism, allowing the server to send messages to the client as well as the reverse. Since I couldn't at first find an example of how to do this using tarpc, I spent a while exploring tonic as an alternative. tonic distinguishes itself by relying on Protocol Buffers to generate the RPC code (while tarpc defines this all in Rust) and by focusing on HTTP/2 as a transport, rather than the more agnostic system tarpc uses. I hadn't worked much with Protocol Buffers since a prototype serialization layer for Analytics data I did at Adobe, but things haven't changed much, if at all.
In the end, though, I did find an example of bidirectional RPC using tarpc. Since it really would be far cleaner at this point to use my existing types defined in Rust rather than externalizing those to the protobuf compiler, I gave up on tonic for this use case and am moving forward with tarpc.
Summarizing
The conceptual shift, summed up for my own benefit: the clients will be "thicker" than they were, taking charge of tracking player observations, and the protocol will be "thinner" than it was, focusing on mutations to each player's cached observations.
That's it for now. Good night and good luck.
Current listening: Revolution-Etude, Chopin; followed by HALOHEAD, by Kunzite.
Pictured: roadmap excerpt
Doing some work on Umpire lately. Umpire is a text-based clone of that wonderful game of yore, Empire: Wargame of the Century. Since 2016 I've used it as a playground for learning Rust and also more recently for experimenting with reinforcement learning.
The objective in this current wave of development is to port the code to a client-server architecture so the game can be played as online-multiplayer. That has me diving into tarpc and tokio for the first time. So far, so good. There's long been a separation between the abstract game logic and the user interface in Umpire, and this maps nicely to the server and client roles. I don't have far to go for a very dumb implementation---one where players have to sit around and wait for each other to take their turns. That fits easily with the current architecture which was designed for hotseat multiplayer, just like the original, but probably isn't the long-term destination.
As always, will keep you posted as things develop.
Pictured: the prior release, Umpire 0.4
Status report on the so-called Git Kit project: it's now switched from plain GTK4 to libadwaita, giving us dark mode support out of the box. I also spruced up the vertical alignment a bit, and learned more about the framework.
Next step: a file-history view....
The last few days I've been focused on putting a Linux distribution on a dog-slow Windows 10 laptop I'm responsible for. (8GB RAM was not enough, it seemed to be constantly swapping to its agonizingly slow magnetic disk drive...)
First, I put Fedora SilverBlue 37 on it, in hopes the immutability would pay off in providing a hard-to-destroy environment for some less tech savvy users. But getting the Broadcom drivers to work required some finagling, which made me think upgrades (every 6 months in Fedora-land) would get messy.
I want to set this thing up and forget about it with the expectation that non-expert users can keep it updated indefinitely. SilverBlue probably isn't it, but it was fun to try, and it did demonstrate that a GNOME 3 desktop runs much better on the HP than Windows 10 did.
So I've found myself back in the world of Debian, Ubuntu, and friends. Linux Mint is the current favorite: Windows-esque, with an LTS support horizon that extends out to 2027, two years after Windows 10's. By 2027, the machine will be someone else's problem. It's not booting as quickly as I expected so... might need to work on that. But otherwise, it seems promising.
Feels like I'm getting some momentum here 
DONE file content displayed in gtk::TextView on click
DONE file list is now backed by gio::ListStore
Content Warning: the screenshot gets a little bit meta; could destabilize your perception of reality?
Have a good one, everybody
Next step: show the contents of the selected file.
And a TODO: eventually move the file list view to something dynamic based on SignalListItemFactory. Otherwise our memory usage will be O(n) where n is the number of paths in the repo.
Jumping back to raw gtk4-rs, got tired of the magic in Relm4.
We're now showing the files in the repo in a static gtk::ListView.
At the mo I'm aiming to recreate something like the git gui UI as a starting point.
Today: deepening my knowledge of relm4. My immediate objective is to add a files list to the sidebar. Once files are selectable, I'll show a file view in the main pane.
I spent a while reading up on Relm's factories, but kept wondering how they relate to GTK's own list views. The Relm docs recommend using GTK directly for lists with many items---I think all the files in a git commit / working directory probably qualify.
Question I've been pondering: if Git Kit (or whatever this thing is called) is going to be a sort of combination of git gui and gitk with a dash of GitLens thrown in, then we'll have to represent the relationship of commits to each other, and of each commit to its file tree, and of each file to the commits that created it.
All at once?
Which is, like, a tall order.
P.P.S. And god, that feeling of writing, like, programs that run and do things on your computer.... I guess I'm old-fashioned. Somehow the web stack doesn't give the same satisfaction.