This page is aimed at answering the following questions from edream on the mailing list:

What kinds of programs can I embed in pida? Can I embed any program? Besides Emacs, what other editors can I embed? Can I embed vim? What makes a program embeddable? Could I embed Leo?

Embedding Generalities

d_rol: Talks about embedding a program (whatever program which is not a simple command line tool) should be split into two subjects: communication with the program, and visual embedding. Communication is possible if the program to embed is open enough, and will depend on the program. And for physical embedding, as far as I know, Pida can embed any program that can be loaded inside a GTK socket.

Emacs

edream: Ali mentioned that pida sends elisp messages to Emacs. Very cool. Isn't this the reverse of what pymacs does? If so, it may be exactly what I need. In any event, exactly how does the message passing work? All details would be most welcome.

d_rol: I have written the Emacs adapter in Pida. It is based on the Vim on, and has less functionnalities currently. If you have not already read the EmacsSupport page, you should.

EKR: Ok. I just reread the EmacsSupport page for maybe the third or fourth time. There are so many details that I don't understand, but I get that a special build of Emacs is necessary so that Emacs can be embedded in pida. I have no problem with that in general, and as a potential user I really don't want to wade through platform-specific build instructions.

EKR: What I *do* want to know is this: how does Emacs work with pida? This is probably totally obvious to you, but it is totally mysterious to me. For example, what's in control, Emacs or Pida? Can I program Emacs from python? If so how? Actually, I'm pretty confused about what it even means to have an editor embedded in pida. What can I do with the combination of pida + Emacs that I couldn't do in pida? And where does pida end and Emacs begin? Please understand that I would like some kind of general answers to these questions before I dive into the task of building Emacs into pida.

EKR: I would be more specific about what info newbies needed if I knew what the info was :-)

I think I would not compare the Emacs layer in Pida with Pymacs. The Pymacs website states that Pymacs aims Python as an extension language for Emacs rather than the other way around. As I understand this, Pymacs' goal is to provide a complete mapping of elisp calls to python code. At the time I wrote the Emacs adapter, I thought it was going too far for what I needed.

Pida makes use of Emacs server for communication. Emacs is started by Pida with a unique server name, so that multiple instances of Pida and standalone Emacs can be started without interfering. Communication between Emacs and Pida is two-way. Emacs to Pida communication consists of notifications. Pida needs to know about Emacs events, like display changes or buffer killing. These notifications are obtained using a small elisp script which is loaded on Emacs boot. The script connects to Pida using a socket and registers some hooks. On some events, hooks are called, and a message is sent through the socket.

Pida to Emacs communication is normally made of orders. Pida commands are translated in a set of elisp commands (like find-file, save-buffer or kill-region) and sent to Emacs using this very socket, with the help of the emacs-client program. I thought in the past we could write the behaviour of emacs-client inside Pida, but it works well in this state for now.

Emacs communication and embedding is the matter of two modules, fairly independent of Pida code. They are source:trunk/pida/utils/emacs/emacscom.py and source:trunk/pida/utils/emacs/emacsembed.py.

EKR: This is actually quite helpful information to have. It gives me an idea of what pida is doing. I suspect that the mechanism for communicating with editors is a really nifty feature of pida. For me at least, when I am learning something (and there are *many* such "somethings" in pida) I want to be told the following, in order:

1. What a feature accomplishes, in general. In this case, pida can communicate with and control multiple instances of Emacs, or other editors, as you describe above.

2. How I, the user, will use the feature. Here I am pretty confused. I know that pida has these capabilities, but I don't know how that will affect me. Do pida commands communicate automatically with editors? If so, which commands do so, etc. Can I write python scripts to control Emacs? If so, must I invoke the scripts directly, or can some scripts run automatically?

3. Once I understand in *general* how I (the user) will use the feature, only then do I want more details. Until I understand in general, the details may as well be written in Greek.

EKR: IMO, these considerations are more important for pida than for other programs, because what's new about pida may be pretty revolutionary. In other words, the cooler the product, the more teaching may be required.

Vim

d_rol: Vim was the first editor to be embedded (please confirm), and is currently the best supported one.

AA:

To embed a program in PIDA it needs two things:

1. A remote protocol of any kind, this can be:
   1. Python (since we just run it in the same interpreter and control it)
   2. A formal remote protocol eg Vim/Emacs
2. (optional) A way of physically embedding the application. This is easy for command line applications, but less easy for graphical applications. Graphical applications should generally be able to conform to the XEmbed protocol, which basically means: tell me which X Window ID (or Window Handle on Win32) to inhabit, and I will do so. Applications such as Vim and Xterm comply with this protocol.

You can embed any editor that fulfils 1, and for niceness 2, but you will need to write support code that adapts the PIDA Editor interface into the remote control of the editor outlined in 1. The PIDA editor interface is simple, and complete, in that it does anything we have needed from an editor, and also it is easy to implement for any editor we have tried.

We have embedded:

1. Emacs
2. Vim
3. Medit (notepad-like)

You could embed Leo if it is remote controllable (which is must be, being Python) and if it is physically embeddable (as discussed).

EKR: Naturally, this gets my attention. However, I have no real idea what it would mean to embed Leo into pida. That's where I am stuck. For example, I just fired up pida on my Linux box. I see the main window and lots of tabs. I choose "new file" and see a new editor. I see a menu. So far, so good. But I am *completely clueless* about how any or all of this would change if Leo were used instead. Would the menus change? If so, would the Leo menus replace the pida menus? (I'm guessing not--but then would the Leo menus appear in addition to the pida menus?) Do you see what I am trying to get at?

EKR: Hmm. Are you thinking of pida as "Eclipse in Python" ? If so, then I am guessing that embedding Leo would amount to repackaging Leo as a pida service. So there are all the issues of having services specify what gets overridden. Continuing this thought, I'm guessing that some parts of pida are intended to remain fairly stable, but perhaps not. Presumably almost anything could be altered, as in Eclipse? Am I getting close? If I am close, then it might be good to mimic the Eclipse docs in your intro. Even the phrase "Eclipse in Python" might eliminate a lot of confusion.

Exactly how does embedding work? What code would I write (in pida) to make embedding happen? If there is a clever or difficult pida module that does the 'heavy lifting', what is it? And how would I use it?

Embedding is a simple process for which there is generally no "heavy lifting". The communication to and from the embedded application is facilitated by the PIDA Service framework, by which any embedded component can provide Options, Menu/Toolbar, Other UI, Events, Actions, Keyboard Shortcuts, and Features. These concepts are explained (in detail) in the service authoring part of the PIDA handbook.

EKR: Well, as my ramblings above indicate, regardless of the programming complexity or simplicity, there is considerable conceptual work that the newbie must do to understand what embedding means. And furhermore, the Service Authoring Guide does not give enough of the big picture for the newbie to understand what the details mean.

EKR: Enough for now. The main point is that I am, at present, totally confused about what pida is all about :-)