Messages

Things to consider:

• You could work for me. I have no money to pay you though.
• You can try to contact people and talk with them. E.g. I did in the past talk with Tsvi, Lucious, Evan, Buck, and Abram online. I guess I did "know" all of them already, meaning I was in a context that made it natural to talk to them. But in principle I think you could reach out to people online and talk/work with them. Probably collaboration comes out of first talking.

Your entire "I want to go into research org thing" would in some sense not be neccesary if you could create research connections this way, and also manage to work on things on your own.

Insight: GPUs are special. They can compute parallelizeable tasks much faster than CPUs, and at the same time have a video output. It is possible to run a computational task, and as it is running visualize the computation efficiently. No memory needs to be transferred to the CPU for this.

Hypothesis: This can be very useful for studying algorithms. The right real time visualization of the right algorithm would constitute a powerful microscope. I expect adding the right kind of interactivity that let's you steer the visualization in real time is an important factor (sort of turning into a game you explore to generate insight).

Is this related to something you are exploring? Evidence: David implementing software resterizers.

Do you know auto-editor? You may find it useful. E.g. the following command takes a video, then cuts out all parts that are silent and where there is no movement. Then in a second pass it speeds up all parts that are silent, which results in speeding up the parts where something is moving but there is silence. This is useful for screencasts, where movements are easy to detect.

auto-editor --edit '(or audio:4% motion:0.0018,blur=0,width=1920)' 2025-01-16_13-42-03.mkv && auto-editor --edit audio:4% --silent-speed 6 2025-01-16_13-42-03_ALTERED.mkv

I talked for 8 minutes with Abram at the AISC opening. I tried to explain why I think programming is very useful. Abram ask me if I have an example of how programatic exploration can be used in alignment research. It's very useful for understanding intelligence I think, but in the following I came up with a setup that might be useful for working on Abram's tiling agent agenda.

Based on our brief conversation at AISC today, I tried to generated a simple setup of how one might use programs to augment how one is doing alignment research. Specifically an environment that seems relevant to the tiling agent problem.

It's a concrete environment, that can be run on a computer, that you could use to explore what kind properties of programs makes a perfect reasoner think that an agent with these properties would be good to create.

A basic property that a tiling agent needs to have is that if I clone myself (or modify myself), that it's actually a good thing. As a toy model we could consider a very simple environment. The environment could be a 2D plane. Squares continuously appear at random locations, and disappear after a set ammount of time.

Each agent in the environment has the ability to copy itself, and the ability to instantly kill an arbitrary other agent.

We start with a single agent. The overall score the agent tries to maximize increases with each square collected, and decreases every timestep by a small amount for each agent on the map. The agent dosn't care about it's own survival. It just cares that throughout time score is maximized.

The environment is such that you need to spawn new agents in order to maximize your score.

If we make the environment small, and have a finite episode length, we can implement a simple action-tree-search agent, that searches through all possible action sequences to determine what to do. Assume that this tree search can simulate the environment perfectly. Each tree search runs to the episode end. It would spawn subagents exactly when this is optimal to do for maximizing the score.

We can now perform multiple experiments in this environment. For example we could hardcode in what subagents can be spawned. Specifically, for each subagent we write some source code manually. We then observe what source codes the optimal tree-search-agent likes to spawn. E.g. the agent that just deletes all other agents and then itself would not be spawned. I predict that three such agents would tile itself. The interesting thing here is that we can now write programs inbetween these two extremes where we are not certain whether they are "aligned". We can compare these algorithms with each other by offering the action-tree-search agent a set of algorithms and see which ones it prefers (which can be situation dependant).

With such an environment we might be able to explore what kinds of structures within the code, are relevant for tiling. What structures makes the perfect action-tree-search planner spawn or not spawn a subagent.

You could try to use Minikanren to search for programs with certain properties. E.g. programs that are spawned or not-spawned.

This is just the first thing I came up with. Perhaps it doesn't work. Likely this isn't the best thing to do. But to me it really feels like that doing something of that nature can be very high value. The idea here is to have the environment a sort of measurement device. Ideally the environment allows us to extract a lot of useful information, that would take a lot longer to obtain otherwise.

Some random thoughts:

• Quines seem related to tiling agents. It may be fruitful to study quines concretely for better understanding how to create a tiling agent.
  • What would a functional quine look like, i.e. a quine that returns a program that has the exact same input output behavior.
  • Can you make a quine that can be "changes as much as possible" while still producing the source code of the original program?
• You could implement a simple logical reasoning system that tries to proof that an algorithm will aways be good, such that we don't need a finite episode length.
• The setup could be extended by parameterising an agent. E.g. the tree-search agent constructor takes as an argument how deep of a search to perform. When the tree-search agent decides to clone itself, it now also computes how deep the tree search of the clone should be. For tractability we could discretise the range of possible values for the search depth paramater.

[Further conversation not exported]

Read a bit about your work. Here are some thoughts.

I did an experiment where I wanted to implement a specific kind of graph algorithm. I had one person talk to Claude and Claude with write most of the code. I basically had a very high-level interface now. I could say "Implement an algorithm that counts how many paths there are that get you from a point a to a point b in a graph?" And this worked very well initially but then it sort of broke. Me and the other person had created hundreds of lines of code at some point. And it sort of worked and it was easy to create it. However, then we wanted to do some more experiments, and Claude did not manage to make the changes correctly. The longer we went along the harder it was to get anything to work at all, until we reached a point where we didn't get the thing to work in maybe 15 minutes at all anymore.

What are your thoughts on this?

Another point: Having a tool where you can dump all your thoughts into one place and then have the AI help you make the relevant connections seems very good. I am wondering how capable an AI needs to be in order to be able to be able to create such a system. Or rather how the most useful system I can create with this scales with the intelligence of the AI.

I don't know:

• How soon we'll get capable enough systems.
• Whether the capability required to make these tools work really well are so dangerous that we are dead already. This is really the crux for me so far.

Also, it seems this research could be very dangerous if it succeeds hard. Good live interfaces could be used to speed up capabilities research. So if this agenda succeeds it would seem you want to be careful about what you'd publish. You only seem to vaguely mention that issue offhand. So I wonder what your thoughts are on ensuring that the research can't be abused?

Non-AI: I am also wondering what you are thinking about non-AI automation tools. I think that basically all software systems are really bad at being good, i.e. if we consider how good something could be, then we are very far away from that. As a random example consider that Obsidian doesn't use UUIDs to link to entries, meaning if you move an entry to a different file all links to that entry will break, making it very hard to freely restructure your knowledgebase.

Do you have any concrete demos of live theory in action? I would be very curious to see them.

There I suggested that the other participant could participate initially in both projects to get more information about which project they like more, and then decide (which still seems potentially the best approach to me).