UP | HOME

Learning Learning: Being an Expert at becomming an Expert

Table of Contents

Learning is extremely important.

1. Explicit Learning for Algorithm Extraction 🔗

The human brain has many low level algorithms for learning. When learning to play Minecraft you can do so without much analytical thought.

However, as Learning Cyberhook via Curriculum Learning demonstrates, analytical learning can be very powerful. Using it you'll be able to learn a lot quicker. But another great advantage is that because I are thinking about the learning process consciously it is a lot easier to introspect and figure out how you are doing it.

2. Learning Cyberhook via Curriculum Learning 🔗

Cyborhook is a great game about movement. I briefly played a level where you need to grapple yourself through a narrow corridor. I was extremely bad at it. But eventually I managed to do it:

The game has a slow motion mechanic that makes things simpler. I didn't even use it. In the end I was able to beat the high score of my brother for this level who played the game for 8 hours, after playing for maybe ~30 minutes. I only played one other level before this, which wasn't about being precise.

2.1. Making a Curriculum 🔗

The reason I was able to do this is that I analysed what was going on. Initially I "just tried" to go through the corridor. But then I realized that I could break down the learning tasks into chunks, creating a curriculum for myself.

Note how there is a first section of the corridor that is straight. I noticed that I wasn't even able to enter the corridor correctly. I realized that you want to have a velocity vector that points slightly up when entering the corridor through the center, such that I would end up smaching into the center of the back wall. This then is pretty close to having a constant horizontal movement, taht you can then carry through the corner.

So now I tried for many minutes to just consintently hit the back wall in the center, getting the entry velocity vector just right.

2.1.1. You need to combine learned Blocks 🔗

Only once I managed to do this consistently did I add a new step. Now I tried to turend around to look at the corner. I immediately failed to now even enter the corridor. Hypothesis: I constructed an temporary algorithm in my brain that is good at hitting the back wall, but when I try to add more to it like looking at the corner it disrupts this algorithm.

I again now tried to be able to hit the wall consistently, before trying to look at the corner again.

So an important part of curriculum learning is that you need to train to combine the individual building blocks.

You can train playing a cord on a guitar in isolation. Once you mastered that you need an additional step of learning to transition from this cord to any other cord.

2.2. Freeing Cognition for the Next Step 🔗

This now felt very different from my inital strategy of "just trying". It felt like I could understand what was going on with my movement. I could see myself going around the corner. Perhaps my brain had more brain power to analyse the situation because the basic movement execution had now become muscle memory. It was though probably also because I trained myself to have the right velocity. I want to throw myself into the tunnel as slowly as possible. Initially I entered it a lot faster which made it harder.

At some point I got a perhaps 30% hit rate of being able to correctly around the corner. Now the next bottleneck was to maintain the correct height. You need to grapple such that you pull yourself up slightly, such that you neither hit the floor nor bottom.

I then goot good enough to be able to do this with <10% chance or so, and each time I managed to enter the big room I tired to grapple towards the portal. After 2-4 attempts I was able to grapple into the portal. At that point I did not try to optimize futher.

2.3. What would be the Next Step? 🔗

However, if I wanted to do this, I would want to do the same thing. Now that I can with low probability can do the entire thing, I need to identify the part that I am weakest at and specifically train for that. That part would be entering the portal quickly when exeting the tunnel. I probably want to train to grapple slightly to the right of the portal immediatly when exiting such that I pull myself straight towards the portal.

3. From the Depths 🔗

3.1. Curriculum Learning for From the Depths 🔗

From the Depths is the most complex game I played. You build combat vehicles out of blocks. There are some "simple weapons" that you can just place, but they are all very bad. Any good weapon will be build out of many blocks. How you build such a gun will determine the guns characteristics. You give the gun a longer barrel by placing more barrel blocks, which makes it more accurate. Everything can be customized. Even the amunition. If you fire shells 5 blocks long, you could say that how much of the shell is charge, and how much is high explosive, EMP, fragmentation, etc.

Sometime before 2023 I played between 50-100 hours of this game. But actually I am still terrible at the game. I am bad at learning it efficiently. How would I do this?

3.1.1. Building a Map of What there is to Know 🔗

I think I can come up with a good strategy for learning the game, because over my previous playtime I already know most of the mechanics that exists. Though I don't understand them. E.g. I know that there is an armor mechanic in the game. If you stack blocks the entire stack of blocks becomes more resistant. But I don't know the details.

If I didn't know the systems in the game, the first step would be to to build a high level map of them. As a first step I might want to skim through all of the different block types that exist.

This would probably invole things like:

  • For each type of weapon, build a minimum working version.
  • For each type of vehicle (tank, ship, airplane, submarine, spaceship) build a minimum version that can move around.
  • For each type of power generation (engine, steam engine, electric generator, etc.) build a minimum working version.
  • For each type of propulsion system build a minimum working version (propellors, rotor blocks for air and water, jet engines, fortress controller.)

For figuring out armor I might want to simply build up different stacks of armor and then shoot it with different weapons. Initially this could just be the characters personal weapons.

3.1.2. What does it mean to learn Efficiently 🔗

What is the best way to learn? It depends on your goal. A good first step is to list out all the things that you could learn:

  • Build the most beautiful ship.
  • Build an asteatically pleasing ship as fast as possible.
  • Build the best long range laser for cutting through layered metal armor.
  • Be good at building a particular weapon system as fast as possible.
  • Become good at using the LUA controller to control missiles.
  • Become good at using the LUA controller to control your ship.
  • Build a ship that has a high ratio.
    • Engine uses less fuel
    • Weapon attack power
  • Build a constant that can move as fast as possible.
  • Build a construct that is as nimble as possible.
  • How to armor something against cram shells.
  • How to use smoke dispensers against lasers optimally
  • How to use shields
  • How to build an engine with as much engine power as possible.
  • How to make a boat go as fast as possible.
  • How to have stable movement in a boat (no tipping over, no matter the engine speed.
  • Determining the right tactic based on the circumstance.
    • Dive when attacked by lasers.
  • How to build a good mothership
    • How to have an effective mothership that can launch figters.
    • How to build new fighters during combat.
  • How to use spin blocks for propulsion.

There are some reoccuring properties we care about:

  • How fast can I do something.
  • How can I do something cheaply.
  • How powerful is something.
    • How powerful is it against specific targets.
  • Good Movement
    • Speed
    • Changeing directions quickly
  • Protection

You could also have a higher level goal like building an alround vehicle that is good in most situations. For that knowing the how to get the above properties would be helpful.

3.1.3. Meta level analysis of the learning procedure 🔗

The primary thing I am optimizing for is building a good relevant model quickly:

  1. Build a high level map that has as few unknows as possible. You don't even need to know what X is at this point, just that it is a thing.
  2. Get a very high level understand of each topic. E.g. that plasma is a weapon type, meaning something that is used to do damage.
  3. Being able to get minimum working versions of things in the map. E.g. you can build a plasma gun that can shoot a plasma round.
  4. Determining what I care about optimizing.
  5. Now you want to optimize to become expert at things relevant achiving the goal. You want to learn whatever topic generates the most utility. You need to take into account how useful a pice of knowledge is, and how much time you need for learning it. This includes:
    • Investigation: E.g. how well do verious types of laser design fare against stacked metal armor.
    • Practice: Be able to quickly build a laser system with specific charateristic, including range, armor penetration, space taken up.
    • Testing: Continually why learning, you want to actually reality test your models. E.g. if your goal is to build an alround good combat ship, you want to actually put it in combat a bunch and observe how it performs.

Of cause while you optimize for expertise, you likely notice that there is a thing that you never thought about before. E.g. you might learn that you can use weapon systems to shoot down missiles. At that point you want to go back and build a minimal working anti missile system.

Basically we still want to do curriculum learning but contrary to the curriculum learning in Cyberhook the space of knowledge is now much larger, and it's easy to not be aware of all that exists in the space. That means initially we need to explicitly optimize to get an overview map.

Another difference is that the goal we want to get to is much less clear. Infact there are many goals we may care about.

Also the value of precise testing setups is much higher. E.g. when building a laser that just points forward and needs no armor, and can take as much space as we want, because we just want to hit a wall of steel blocks to test out different laser designs in terms of how much of each compontent they have, then such a laser is much easier and faster to build. It also makes it easier to see what components the laser is composed of, and allows for easier modification of the components.

3.2. Epistemic Investigations in From the Depth 🔗

In from the dephs I can build many different kinds of constructs, and gather data about them. Consdire that I am building an advanced cannon. Now I'd like a model that tels me about:

  • Cost
  • Turn Rate
  • Engine Efficiency
  • ROF (rate of fire)
  • Accuracy
  • Damage
  • Penetration
  • Armor (How well protected is the cannon)
  • Dodge
  • Profile (how much is over water and can be hit)

These are the high level properties we care about. For now let's just consider what lower level variables influence the properties of fire rate:

  • ROF
    • Ammunition Feeding Speed
      • Shell Design …
    • Ammunication Loading Speed
      • Shell Design …
    • Barrel Cooldown
    • Barrel Design
      • Is railgun?
      • Bore Excavator?
      • Multibarrel?
    • Shell Design
      • Shell Diameter (gauge increasers)
      • Shell Length (loader and magazine length)
      • Shell Modules

We need a model that tells us about how these variables influence the target variables.

If I want to maximize the ROF I when building a cannon, if I know the shell, diameter, and charge, how many coolers I need in order to have the gun shoot at max ROF (at the autolading speed).

We can do the same for accuracy:

  • ROF …
  • Barrel Design
    • Barrel Length
    • Muzzle Breaks
    • Bore Excavators
    • Railgun attachments
  • Recoil
  • Recoil Absorbtion

Note how ROF is an important property for accuracy because each firing generates recoil, which can accumalate and decrease accuracy.

To create such a model, perlian causal modeling seems relevant. Also looking at special case deterministic models seems relevant.

3.2.1. Building an Advanced Cannon Model 🔗

For practice let's build a causal model of advanced cannons.

  1. We want to lay out all the different properties that a cannon can have.
  2. We want to create a model of how these variabels influence each other.

To make things simple we can first look at a subset of properties, assuming that all other properties are fixed. Let's model the relationship between "minimum number of coolers we need for systained fire" and "firerate", assuming all other properties like gauge, barrel lengeth, shell length, shell modules, etc. being constant.

In practice that means that we are only allowed to modify the gun by adding collers and autoloaders.

  • Note how now have made the implicit assumption that it doesn't matter where the cooler is added. Perhaps a cooler is more effective if it is closer to the firing pice. We can emperically test if this invarient is true.
  • A similar thing holds for how the autoloaders are located.

When I say autoloader, I am actually refering to a specific construction of autoloader, magazine and ammo feeder. Whatever design we end up using (e.g. beltfet-autoloader, or normal autoloader with n magazines of length m, with k ammo feeders) we should always use the exact same design.

3.3. Explicate: Design Before Implementation 🔗

According to Rich Hickey design is very importent. E.g. if you want to build to build a rocket, it's much faster to create technical drawings laying out the design, than it is to build the rocket. Explicating in a format that is easy to create and change allows you to notice and correct flaws quickly.

Without explicating many flaws would not be discovered, and discovering them directly creating an implementation is often much much slower. And in the case where your rocket explodes much more expensive.

Making designs on paper (many other media exist) even for From the Depths where the iteration loop is pretty fast and pretty cheap, can benefit from design.

Here is an example. I had the basic idea that I want to have an advanced cannon wich a turret cap as small as possible, with all the internals being hidden below the deck of the ship. To explicate I created the following figure.

20250108_153516.webp

Figure 1: Gun Mantlet Design

I then went on and created multiple designes for how to lay out the internals of the turret.

20250109_180834.webp

Figure 2: Feeder A: Vectically Stackable ammo Feeding Mechanism

20250109_180838.webp

Figure 3: Feeder B: 6x6 ammo feeding vertical array; non-stackable; maximizes space usage

Then something wierd happened. I played from the depth in order to implement and test the design. However I endeded up doing to implement a completely different kind of turrent design without any reason really. Possibly my brain just wanted to play the game without thinking, as that's the default strategy, and just in the moment ad hoc do things. The main problem of this turret was that it could be shot off extremely easily, likely because it was a design where the entire turrent (including internals) sits ontop of the ship. Exactly the problem my design tried to avoid.

After noticing how the ad-hoc turret design completely failed I set out to implement a variant of the previous design. I ended up with a much more heavily armored turret cap, and created the following layout for the internals, ingame game (the drawing was created afterwards).

20250109_193706.webp

Figure 4: The top layer of a stackable array of turret internals. It's self contained in that all components required for the turret are present such that any number of stacks do not require any extra configuration.

This layout was then stacked between 20-35 times to create an extremely expensive and powerful single barrel cannon, with a range so large that it can shoot godly spacestations within 1 minute, from a distance so far you can't even the craft unless you zoom in maximally.

Because the internals sit in the bow of the ship, simply by adding more armor to the ship we increase the armor of the cannon (though this has not been tested).

3.4. Reverse Engeneering The From the Depths Construct File Format 🔗

Here is an example construct file: 

{"FileModelVersion":{"Major":1,"Minor":0},"Name":"test 1 wood","Version":0,"SavedTotalBlockCount":1,"SavedMaterialCost":1.0,"ContainedMaterialCost":0.0,
"ItemDictionary":{"322":"9a0ae372-beb4-4009-b14e-36ed0715af73","721":"e63040c9-0027-4fd3-be30-67fe3e950140"}

,"Blueprint":{

"BlockIds":[322,322],
"BLP":["0,0,0","0,0,1"],
"BLR":[0,0],"BP1":null,"BP2":null,
"BCI":[0,0],"BEI":null,

"ContainedMaterialCost":0.0,"CSI":[0.0,0.0,-1.0,0.0,-1.0,0.0,0.0,-1.0,-1.0,0.0,0.0,0.0,0.0,0.0,-1.0,1.0,1.0,0.0,1.0,3.0,-1.0,-1.0,0.0,-1.0,1.0,1.0,0.0,0.0,0.0,0.0,0.1,0.0,0.1,0.1,1.0,1.0,3.0,-1.0,0.0,-1.0,0.0,-1.0,1.0,0.0,-1.0,0.0,0.0,-1.0,0.0,-1.0,0.0,0.0,-1.0,1.0,0.0,-1.0,0.0,0.0,500.0,-1.0,12.0,0.0,0.0,0.0,200.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,1.0,1.0,-1.0,-1.0,-1.0,-1.0],
"COL":["0.01176471,0,0,0","1,0,0,0.5","0,1,0,0.5","0,0,1,0.5","0.1,0.1,0.1,0.5","0.3,0.3,0.3,0.5","0.5,0.5,0.5,0.5","1,1,1,0.1","1,1,1,0.5","1,1,1,0.99","1,0.9215686,0.01568628,0.99","0,1,1,0.99","1,0,1,0.99","0.6588951,0.2146527,0.9548753,0.6416647","0.6885689,0.2575601,0.7009315,0.3807098","0.9453253,0.7970616,0.451286,0.7535138","0.8480695,0.82942,0.4679194,0.971258","0.5534453,0.9874221,0.2497269,0.9798778","0.4714386,0.7396991,0.4725898,0.2763405","0.443287,0.4070382,0.3845369,0.9562324","0.1433013,0.7042804,0.7507681,0.5912738","0.1257714,0.03002892,0.04892975,0.5434183","0.3543451,0.117049,0.7015883,0.6166628","0.4962368,0.02548637,0.9717376,0.8993787","0.05592966,0.0225,0.1246448,0.8892872","0.6427358,0.1501085,0.8460484,0.541915","0.5945613,0.4104804,0.9649957,0.9090848","0,0,0,0","0,0,0,0","0,0,0,0","0,0,0,0","0,0,0,0"]

,"SCs":[],


"BlockData":"",
"VehicleData":"8FsKAQAAAAAOAAAAAAAPJwAAAAAA0AcAAAAAAJ9xBAAAAAAADgAAAAAADycAAAAAAKCGAQAAAABXLgAAAAAAAAcAAAAAAAAAAAAAAABHKQEAAAAAABUAAAAAAAAAAAAAAABzJwAAAAAAZAAAAAAAAD41FAAAAAAABwAAACMAAQAAAAAAAAoABM3MzD0LAATNzMw9DAAEzczMPQ0ABM3MzD0OAATNzMw9UkMAAAAAAAAHAAAAAAABAAAAAAAAwX1WnwAAAAAHAAAABwAAAAAAAAAAAAAEAQAAAMRKAAAAAAAABwAAAAAAAAAAAAAAADntBwAAAAAABwAAAAAAAQAAAAAAALd02QAAAAAAHAAAAAAA6UgBAAAAAOhIAQAAAAAxbhYAAAAAMG4WAAAAACveCgAAAAAABwAAAAgAAQAAAAAAAAAAAQEBAAEBWaAAAAAAAAAAAAAAAADQq2UAAAAAAA4AAAAHAAAAAAAAAAABAAAAAAcAEQAEAQAAAMInYQAAAAAABwAAAEYAAAAAAAAAAAAABAAASEIKAAQAAMjBDwAEAABcQhQABAAA+kMZAAQAgDtFMgAEAEAcRjcABABAHEYeAAQAAIA/IwAEAACAPygABAAAoEEyjlAAAAAAAAcAAAAAAAAAAAAAAAAuywQAAAAAAAcAAAAAACSIAAAAAAAS6AEAAAAAAA4AAAAAAA8nAAAAAACghgEAAAAAoflWAAAAAAAOAAAAAAAPJwAAAAAAoGNvAAAAAA=="

,"designChanged":false,"blueprintVersion":0,"blueprintName":"test 1 wood",

"SerialisedInfo":{"JsonDictionary":{},"IsEmpty":true},"Name":null,"ItemNumber":721,

"LocalPosition":"9967.509,-2.5067,10376.55",
"LocalRotation":"0,0.06097321,0,0.9981394",

"ForceId":1361640007,"TotalBlockCount":1,"MaxCords":"0,0,0","MinCords":"0,0,0",

"BlockState":"0","AliveCount":1,"BlockStringData":[],"BlockStringDataIds":[],"GameVersion":"4.2.3.0","PersistentSubObjectIndex":0,"PersistentBlockIndex":0,"AuthorDetails":{"Valid":true,"ForeignBlocks":0,"CreatorId":"b2eb5019-c74c-45f6-b7a7-a5de2fbe19d8","ObjectId":"f4ff992c-44d6-403b-a3b1-6827942f977d","CreatorReadableName":"AIKILLSUS","HashV1":"53c7db840b4f499473d2fead6540efc2"},"BlockCount":1}}

The important parts are 

{...
"ItemDictionary":{"322":"9a0ae372-beb4-4009-b14e-36ed0715af73","721":"e63040c9-0027-4fd3-be30-67fe3e950140"}
...
,"Blueprint":{
  "BlockIds":[322,322],
  "BLP":["0,0,0","0,0,1"],
  "BLR":[0,0], ...
  "BCI":[0,0],
...
}}

The BlockIds, BLP, BLR, and BCI are arrays all arrays of length n, where n is the number of blocks in the construct. At an index i the BlockIds array says what block there is (presumably this is looked up in the ItemDictionary), BLP says where it is (in local vehicle coordinates), (hypothesis) BLP indicates the rotation. BCI has an unknown purpose. If the arrays don't all have the same length the blueprint will not load ingame.

4. Having Ideas 🔗

Discovering new strategies that actually work well is a major way in which you can become better.

Put yourself out there:

In general when trying cheesing games in creative ways seems to exibit the kind of exploratory behavior neccesary to also in reality find stuff that works. Cheese reality as hard as possible, cheat as hard as possible. By that I mean find the really "dumb" simple strategies that seems like they couldn't work because they are so effective, but that actually do outclass complex solutions.