The ghost in my sh.el

(eva “say hello world”)

We are in the middle of a once-in-a-lifetime revolution. AI is here and more is coming. If you’re also an Emacs user, you’re watching this revolution unfold from within your Emacs buffers, with the aid of packages like copilot.el or the amazing gpt.el. I continuously see more and more of my workflows aided by these models.

But what if we could go further than calling 3rd party stock models. What if we could go A LOT further?

To understand what EVA can be, follow me into a short exploration of the idea.

Introducing Emacs Virtual Assistant, short EVA. The first AI system intended to work alongside you within Emacs.

Premise

As AI becomes more prevalent and capable, it will handle an increasing amount of execution. Leaving us humans to drive intent and to judge design. A symbiotic relationship will form between humans and AI systems. Human intent will drive agent execution.

This process of co-creation needs to be facilitated in a suitable environment. Both participants of this relationship, human and AI, have their own roles and needs. The AI needs methods for reasoning through instructions and problems, and tools at its disposal to do the actual implementation. And we need to access to the reasoning and execution. To judge valence, provide feedback, and steer. Reducing friction in this interaction will be critical, requiring fast feedback loops. And Emacs provides the perfect environment for this.

Emacs special sauce

Emacs provides three key ingredients that make it the ideal environment for human-AI symbiosis. It is entirely text-based, is based on an interactive runtime, and has a massive ecosystem.

Text

Language models, Emacs, Humans, and the subject matter we operate on (code, org-mode, webpages, emails, etc.) share text as their common denominator. This greatly reduces friction. In Emacs, everything is controlled entirely through text. Specifically by sending Elisp function calls to the Emacs interpreter. We load the subject matter of our work, as text, into emacs and call functions on it. From opening the file viewer to summoning a shell, every keystroke is a function call. And EVA can call the exact same functions by generating text which is piped into the interpreter, thus controlling Emacs the same way we do.

Runtime

Emacs’ interactive environment allows EVA and us to inhabit the same space while working. To draw on a shared toolset in the form of the functions that are defined by Emacs, installed packages, or by us. As well as a shared context in the form of buffers, files, and processes.

We load the subject matter into the context and call functions on it to interactively bridge the gap between state and goal. From the AIs perspective, Emacs is essentially an interpreter with an editor, an application platform, and an operating system built in. All of which can be read and controlled entirely through uniform text-based instructions. This fundamentally sets Emacs apart from other editors, that are pure point-and-click text manipulators. Their instruction set is either not accessible or non-existent. It also sets Emacs apart form a Python or Jupyter based environment, which doesn’t provide the combination of shared, interactive runtime and text-based instruction set.

Emacs is a living, breathing ecosystem that both the human and the AI agent can co-exist in simultaneously. That is a very powerful paradigm for co-creation with an AI.

Ecosystem

With Emacs, we can do anything that can be done with text on a computer. For any job to be done that involves text, there is a package for it. This is a testament to the breadth and depth of possibilities the Emacs enables. And EVA immediately has the entirety of these capabilities at its fingertips. To steal a Ben Thompson phrase, Eva is hijacking existing infrastructure.

Whether you’re coding in Python, writing LaTeX documents, or managing your life through Org-mode, there’s likely an Emacs package that’s got you covered. The inexhaustible library of functions and packages means that EVA can be trained to assist with virtually any text-related task you can think of.

Thats the three reasons why the environment Emacs provides is optimal for collaborative creating with an AI. Text, runtime and ecosystem.

The Ghost in my sh.el

So, how does EVA leverage Emacs’ text, runtime, and ecosystem? To answer that in one sentence: You have an ongoing conversation with EVA, in the same space that the execution happens in, using a shared toolset.

Execution

When you call (eva ""), EVA generates Emacs Lisp code that’s executed by the Emacs interpreter, directly affecting the editor’s state. She can work directly on the project files and adjust the code as needed. She can read the content of any resource, shell, or process within your project to better understand the context. If a specialized task arises, EVA can spawn sub-agents focused on reasoning, planning, or solving problems. If she needs clarification, EVA can initiate a chat buffer to ask you directly. When you’re debugging a problem, she’ll have access to all the same information you do, and then some. You could even call her on a cron job to add some initiative on her end, or to automate routine tasks.

Observability

One point on observability here, which I’ll come back to later. This immediacy of writing directly into the editor provides us, human users, with direct visibility into her reasoning and execution. It keeps feedback loops tight and aligns the model to our intent. We will most likely never be able to look into a model and understand what is going on. But tracing an AI’s actions and reasoning in the text is well within our abilities.

Ongoing Conversation

So, what’s it like to co-create with EVA in real time? Imagine a workspace where you’re not just coding directly but actively conversing with EVA. She takes on the heavy lifting of execution, allowing you to focus on higher-level thinking, design, and decision-making. The project you are working on is loaded into Emacs, creating a shared context for both of you. You can monitor her actions in real time, step in to correct course if she veers off track, or provide additional context when she hits a roadblock. It’s not just collaboration; it’s a real-time partnership. It’s like pair programming, but your partner is a machine that can think and act.

Imitation

Let me present one final thought: If we employ Reinforcement Learning, EVA can learn by imitating us. EVA could predict our subsequent actions, effectively learning from the delta between her prediction and our actual action. This would turn every session with EVA into not just a collaboration but training.

Future

Microsoft will build out Visual Studio Code and Windows, “Open”AI have launched their MVP with the plugins marketplace, and countless startups will join the race. However, all of these act as intermediaries, sitting in the middle, taking control and introducing friction.

AI is still in its early days, and the amounts of compute required are substantial. This means we need to maximize the utility of constrained resources. We need to remove friction for this reason. Another reason is that there is a prevailing fear that AI will destroy society. While we are at a stage of AI being glorified autocomplete, I currently see no better way of forcing alignment than keeping feedback cycles small and having complete visibility into the reasoning and execution of the model. Additionally, as a creator, I want to have the closest possible relationship with this new tool. To learn its ways and to get a strong intuition for it. Lastly, cold hard evolutionary dynamics will drive us to optimize. We want to be and are required to be at our most powerful. I want the power to create.

As highlighted in the leaked Google memo, open-source is the winning horse in this race. After 45 years of existence and continuous evolution, Emacs suddenly finds itself in a prime position to be ground zero for a wholly new paradigm of computation, and maybe, of life. I think we need to come together and build this.

Conclusion

In summary, we are a long way away from the ultimate vision I’ve laid out above. But that is OK. Rome wasn’t built in a day and neither was Emacs. If I am right, EVA will one day be as significant to Emacs as org-mode.

This will require time, effort, creativity, innovation, problem-solving, money, and much much more.

Ultimately, it demands what we humans uniquely bring to the table: intent.