/ Hathaway Weblog / Zope 3 IDE Considerations

Shane :: Zope :: April 02, 2005 # Zope 3 IDE Considerations

For a while now I've considered starting a Zope 3 IDE (integrated development environment.) It could provide wizards for creating content classes, diagrams for manipulating configuration, and a big red restart button. However, I don't want to start from scratch. I want to plug into an existing IDE.

Now I need to choose which IDE to extend. Unfortunately, there are now so many IDEs that support Python that it's hard to choose one. Here are the current Python-aware IDEs that I know of:

Here are the criteria I think I should use for selecting which IDE to extend:

  • the file dialog must follow certain rules or must be replaceable with a dialog that follows those rules
  • the code must be clean, maintainable, and modular
  • cross-platform
  • the IDE must support wizards and pluggable editors (such as a graphical diagram editor)
  • I might consider proprietary IDEs, but open source is much more attractive

Can anyone suggest which IDE might be best to extend? I should probably ask the Zope 3 and Python mailing lists. A prerequisite is that I must start using the chosen IDE for daily work.

Comments

Ian Bicking (April 03, 2005 20:25)
Something that would be useful in any environment with the potential for stale code, would be a thread that scanned all the "loaded" files (in Zope 3 that would include ZCML, but in other systems that might only be files in sys.modules), and warn you if any had been updated. I like restarting in that case, but if that's too aggressive (e.g., you can't do a restart quickly) you should at least put big bold warnings on every page. At least that's a really easy way to improve the experience in long-running servers.

Joel Lawhead (April 03, 2005 21:21)

The front runners for a Zope IDE are Boa Constructor and Eclipse. Both are rich IDEs that provide some limited support for editing Zope Page Templates already through ftp and WebDAV.

Of these two options Eclipse stands out for several reasons.

  1. Eclipse provides a rich, cross-platform IDE for several languages including Python, Jython, C, Java, and markup languages like HTML and XML. Most Python programmers must work in more than just Python sometimes if not often. Even if you work strictly in Python you will probably write a C extension for Python sooner or later.
  2. You can create powerful plug-ins for Eclipse to create any type of dialog or wizard you want using Eclipse itself to do the development. Plug-in support includes an entire framework for diagram editors of various types.
  3. Both Boa Constructor and Eclipse support CVS but Eclipse also supports Subversion (via Subclipse)
  4. Eclipse has existing models for integrating development for an application server - most notably Sydeo's excellent Tomcat plug-in.
  5. Eclipse, like Zope, has the best of both worlds. It is an Open Source project with a strong community plus the backing of a corporation dedicated to its development.
  6. The Eclipse Updater is excellent for updating all installed plug-ins and the IDE itself with point-and-click ease.

Boa Constructor is an excellent Python IDE that is extensible and used by many people for Zope work. But having used both Boa and Eclipse extensively I think Eclipse provides a better platform for building a Zope IDE with advanced features. I'm sure there are plenty of good arguments in favor of other IDEs and against Eclipse but Eclipse has proven to be powerful IDE for many languages and application servers and is well suited to support Zope 3.

sake (April 04, 2005 03:23)

What I like about developing with Zope 3 is that my logic is almost completely isolated. I don't need any IDE to work with python code. Emacs, doctest, unittest, pdb, print statement do the job perfectly. What make developing zope 3 package different from plain python package is the zcml. What I really need is a fool-proof way to author zcml files.

I have this wild idea for a while.. I'm not sure what exactly how it will be like .. but....

"Why not make Zope 3 IDE with Zope 3 itself ?"

Authoring all zcml and zpt files in local disk through browser. Make a zope 3 product that analyse python source code and make zcml authoring just a matter of point-and click. The zpt files could be author with any fansy html editor and update through webdav or ftp. This way everybody can continue using their favorite python/html environment and while enjoy the elegance of Zope3.

Wouldn't that be cool ?

Shane Hathaway (April 04, 2005 12:54)

Ian: detecting code changes is definitely doable and a big win for a low price. I implemented that functionality in Zope 2, although I now believe it was mistake to try to reload code when changes were detected. It would be much more useful to either warn the developer or restart the process automatically.

Joel: Both Boa Constructor and Eclipse carry a lot of baggage. Eclipse does have the perspective feature, which hides much of the baggage, but the baggage is still apparent from the giant menus and configuration system. Also, Eclipse plugins really ought to be written in Java, not Python, and I don't want to get buried further in Java.

sake: Zope users have long wished for a browser-based IDE. Many have tried to start such a project but didn't get very far. I can think of several reasons why they have failed.

First, former IDE attempts let security concerns impede progress. Zope's general security rule is that nothing you do in the browser writes files outside the database. The IDEs I've seen tried to fit that model, but it's much easier to write an IDE if you simply break that rule.

Second, some IDEs run in-development code inside the IDE. That's a bad decision. Developers need to be free to change anything they like without risking breaking the IDE. Thus even if the IDE is browser-based, it must run in a separate process from the user's code.

Third, a lot of things that are easy in a rich GUI are still difficult in a browser: syntax highlighting, clean dialogs (such as the file dialog), UML-like diagrams, and space-efficient widgets (such as tabbed windows, tree controls, and lists with sortable/resizable columns.) You can't give up those things without paying a lot of usability penalties.

Stephan Deibel (April 04, 2005 16:25)

Well, as co-author of Wing IDE I'm biased so I won't weigh in on which is the best IDE choice. All I'll say there is that you can get access to the full source if you would like to take a look at it.

In general: Why not write as much modular support as possible so that it can be plugged into multiple IDEs? I know this isn't trivial to do for some things, but seems worth trying where possible. Wing IDE has a bunch of Zope2 specific support and we want to add Zope3 support as well. We would probably contribute to any open source components you write that we can plug into Wing IDE. You'ld probably get others helping as well, although I understand you need to balance the overhead of making this modular w/ any benefit you may get from doing that.

In any case, good luck with it!

Shane Hathaway (April 04, 2005 17:42)

Thanks, Stephan. The difficulty in making such a thing modular is the GUI toolkit. Almost everything I'm thinking of is toolkit-dependent. Look at how many toolkits different IDEs use:

  • Boa Constructor - wxPython
  • SPE - wxPython
  • Dr. Python - wxPython
  • Eric3 - PyQT
  • KDevelop - QT
  • Eclipse - Java SWT
  • Komodo - Mozilla
  • Wing IDE - ?
  • (X)Emacs - X11 / curses
  • Idle - Tkinter

PyGTK and Java Swing are conspicuously missing from that list. I guess I should include Glade and JEdit. There's probably something brewing in the Mono community as well.

Since modular won't work, I can't move forward until I can choose one IDE. However, I don't feel comfortable working in any IDE but XEmacs due to the file dialog. I hope someone will step up to the plate and make their file dialog pluggable. If I could adopt an IDE, I would extend that IDE.

sake (April 04, 2005 21:17)
First, former IDE attempts let security concerns impede progress. Zope's general security rule is that nothing you do in the browser writes files outside the database. The IDEs I've seen tried to fit that model, but it's much easier to write an IDE if you simply break that rule.

Nope. There are no such rule. The only security rule zope has is that you cannot perform any secured action unless you are authorized with proper "permission"

Second, some IDEs run in-development code inside the IDE. That's a bad decision. Developers need to be free to change anything they like without risking breaking the IDE. Thus even if the IDE is browser-based, it must run in a separate process from the user's code.

That's what I mean. Zope3 instance which manipulate other zope3 instances.

Third, a lot of things that are easy in a rich GUI are still difficult in a browser: syntax highlighting, clean dialogs (such as the file dialog), UML-like diagrams, and space-efficient widgets (such as tabbed windows, tree controls, and lists with sortable/resizable columns.) You can't give up those things without paying a lot of usability penalties.

Browser interface is suck for almost ANY kind of application, IMHO.(I really hate when I'm typing in this ridiculous box to post this comment) But it is easy to develop application for, and is widely adopted already. Although usability is sacrificed, very often, it's worth to have something working sooner.

BTW, "tabbed windows, tree controls" -- can be accomplished in recent DHTML with the same ,if not better, usability as in rich GUI.

I have UML in my head while I'm design and develop. I draw UML on paper, and sometime in diagraming tools, for communicate with people and documentation. I don't need tools to synchronize UML with the code (if that's what you mean by UML). And I don't even think that's possible in dynamic language, without sacrifice some flexibility, of course.

Perhaps, what I actually need is too little to be called IDE and out of topic for this discussion. (and perhaps something called IDE is too big to be a topic for any discussion). I just need a simpler zcml manipulater/validator, and some kind of zope3-instance manager (move folders & files around, run tests, pychecker, generate&browse pydoc)

Above all, I don't want to leave my emacs. ;-)

Looking for clean and extensible framework in python ? Isn't it named Zope3 ?

Peter (April 05, 2005 10:49)
My advice is to start from scratch and use as much as you can from the others.
  • Boa is done in wxpython but it uses the old namespace and lately it had some problems moving to 2.5.
  • Eclipse is written in Java and it would be better to have an IDE expandable in Python. Eclipse is well designed tho. The best designed IDE I've used.

If we could reimplement parts of Eclipse in wxpython....

What the python community needs is a designer... there are a lot of programmers but very few experienced designers willing to work on a OSS python IDE. MVC has been around forever but it is seldom implemented the right way.

I would work for a well designed IDE implemented in wxpython! if someone could serve me small enough tasks I would invest some time to try and solve them.

I even tried to implement my own IDE but... I gave up very quickly... it was more than I could handle from a design point of view.

Dagur (April 05, 2005 12:06)
You should also consider Newedit http://newedit.tigris.org/

David Ascher (April 05, 2005 12:37)
Regardless of what you end up choosing, I think that a great intermediate deliverable would be a list of requirements for what an IDE for Zope3 should look like. That way, whether you do it, or others do it, or IDE vendors (such as ourselves) do it, Zope users get the benefit of a feature set that makes sense from the perspective of a Zope user/developer.

Davy Mitchell (April 05, 2005 13:34)
Dr Python is very good. Lightweight and expandable. The development also moves fast. Not sure about the file dialog (it has a custom one). It runs well on 128MB on Python 2.4.

Josef K. (April 05, 2005 14:27)
When it comes to GUI, Boa Constructor is by far the best editor.

Shane Hathaway (April 05, 2005 14:42)

Peter: I think you're right. It might be best to start from scratch and steal^Wborrow from other projects. I like Boa Constructor and Dr. Python on the surface, but the code seems messy.

David: Also great advice. I should fire up Qt Designer and start drawing pictures. Thanks!

Ian J Cottee (April 05, 2005 17:41)

Whereas in a perfect world you'd have your IDE written in python it seems to me that you are going to have to duplicate a hell of a lot of functionality that would already have been covered by other projects. Think of all the things you are going to need to provide in your run of the mill Zope project. Python, XML, Javascript, XHTML, TAL, TALES, ZPT, CSS, SQL etc etc etc Plus then your CVS/SVN/FTP/WebDav support. Plus whatever new things pop up in the coming years. I think any project which would aim to handle all of the above for an audience which is pretty limited would run out of steam pretty quickly. And if I can't do it all in one environment then I'm less likely to adopt the IDE for my normal work.

The great thing about Eclipse/Emacs is that you can use it for everything. Well at least I use Emacs for everything and I am trying to retrain myself in the way of Eclipse although it could do with a bit more stability in my experience. I'm productive in both with Python despite neither of them being written in Python.

just my pennies worth

Ian

Dan (April 05, 2005 19:25)

The code for DrPython is definitely messy. (I wrote it). (That being said, I could/should work towards cleaning it up.)

I currently use drpy to work on Zope for my job. When I say this, I mean I develop Zope products in drpy, and then deploy them (svn, custom script, putty, depending on the target).

I would definitely be willing to work to make the file dialog play well with others. Especially since I have a vested interest, given that making my editor more Zope friendly would make my job easier.

At the moment, I am still in the moving process, but I am hoping to be moved in a week or so. Also, 3 students at U Toronto have be doing some amazing work, and their release (which should also happen soon) will form the basis of the new branch of drpython. When both of these events are complete, development on the next version will begin.

Best, Dan Pozmanter

PS: It would be especially cool to add a zope toolbar, with the restart bit, error_log, and the ability (maybe a drop down?) to refresh particular products.

Ian Bicking (April 06, 2005 01:12)

I missed this talk on Envisage, and the notes aren't very specific, but it sounds interesting and along these lines. I keep wanting a New Better Emacs, and maybe this has something to do with that...

http://www.python.org/pycon/2005/papers/81/PyCon-2005.txt

Personally I think Emacs' GUI is still top-notch, better than "modern" GUIs that were designed for a new-computer-user audience that hardly exists anymore (and certainly shares few traits with a typical programmer). Maybe a pervasive plugin architecture -- which Envisage claims to have -- would allow a programmer to plug in a programmer's UI (though I'm skeptical such a level of pluggability is really possible -- just like you can't paste a normal GUI on top of Emacs, you can't paste an Emacs-like interface on top of a normal GUI).

Ville Vainio (April 06, 2005 08:04)
Apparently Jython isn't dying anytime soon (it's being updated with new Python features) - wouldn't the best long-term gain be creating a way to write Eclipse plugins in Jython, and do whatever is necessary there? I believe this would make Eclipse the "better emacs", provided that the plugin API wouldn't be too unwieldy...

Shane Hathaway (April 06, 2005 09:47)

Dan: it's great to hear from you. I think DrPython has the cleanest UI of the lot. If the code were clean to match, I think you'd see a lot of new contributors.

Ian: Emacs is good for editing text, but what I have in mind is more graphical. Thanks for the link to Envisage.

There is clearly no emerging agreement on editors or IDEs, and that's not a problem I really want to fix. So if this idea is going to succeed, I need to reduce it to a Zope development assistant that integrates with virtually any editor or IDE. It'll have to run as a separate process and talk to the editor / IDE through sockets.

Independently of that project, I'd like to use an editor written in Python so that I can change it relatively easily.

Andy McKay (April 06, 2005 10:35)

Writing your own is a recipe for a lot of work, working on a plugin for other frameworks will be harder as you go figure out all the API's, but it will allow for a better solution in the end.

I would follow David's advice, get a list of things you would like and prioritise them. Make as much as you can non-IDE specific and things will be on the way :)

To through my two cents in FTP or WebDAV is of no use to use because we develop solely on the filesystem with only the rare exception.

RunLevelZero (April 06, 2005 11:00)
DrPython has my vote. Development is constant. It is lightweight and fast at least for me. Works on OSX, Linux, and of course Win32. The most customizable IDE out there. The developers is very open to new ideas. The new version with major feature upgrades will be great. It's already pluggable but the code could be cleaned up a little as Daniel himself has said.

vegetax (April 06, 2005 22:21)
look at gmail and http://maps.google.com/. amazing things can be done in the web if you are a master in javascript.

Douglas Beethe (April 07, 2005 03:07)

Interesting question, and one which just keeps coming up in one form or another within the Python community. In my experience, the IDE does not matter a lot if you are doing trivial scripting, and usually is more hindrance than help. That said however, when you get into serious app's with significant class hierarchies, multiple threads, and distributed operations, the development environment can make a big difference -- usually boils down to the quality of the debugger and its ability to deal with child threads and remote platforms. Having tried many of the IDE environments on your list, I have found that for this type of app work, the commercial Komodo product is head and shoulders above the crowd.

Have done a lot of work previously with Java/Swing for the desktop app and web applet arenas, but deplore its threading model -- never quite predictable. Most of my experience with Python has been in the world of web apps (and the ever present question of which framework to build on), and I have not found any of the development environments you noted here to be much help for the GUI portion of the apps. In any case, I am certain that your target app/GUI environment will have a big impact on your decision. If you are looking to extend an existing IDE that might excuse just about any commercial product from consideration ...

Tom Dossis (April 10, 2005 21:05)

How about developing a rich GUI presentation layer for Zope3? This would benefit a much greater audience than an IDE for developers only.

Perhaps an initial application could be a alternative rich gui zope to the current browser. From this base you could add in desirable developer features. Those who prefer, could continue to use their favourite editor/environment, and use the alt zope app offering(s) to replace or compliment the browser interaction. This approach could also provide a Z3/GUI application framework out of the box (Z3 schema and rich-gui widgets included), so at the very least you now also have a functional rich-GUI based content management system out-of-the-box. On top of this you could provide Z3 users (non expert developers) drag-n-drop (or other more accessible) ways to setup their own content types.

I'd go for broke and utilise ZEO to bring the Z3 packages directly in the process space of this model. This way I could explore, as simply as possible, the application opportunities this would present.

99hats (April 12, 2005 18:34)

I don't think it's just hype. See Ruby On Rails. It looks like THE killer web app IDE.

http://www.onlamp.com/pub/a/onlamp/2005/01/20/rails.html

http://www.rubyonrails.org

Gabe Rudy (August 12, 2005 10:47)

I would agree with Tom,

A Z3/GUI framework would be a great recourse for those pioneering GUI python programming on large/enterprise level projects.

While I'm working on a PyQT process managment project for our sales team, I keep peeking around for popular (non-beta) python application frameworks that would work for a GUI app (also has a server backend, may have a web frontend in the distant future).

The closest thing I've found is PEAK (http://peak.telecommunity.com/) but Zope would far exceed it in its documentation and maturity.

As far as IDE's go, I'd vote Eclipse for its popularity and expandable architecture. I've been considering working on a Eclipse plugin to do live error detection to the level of pylint and in the style of IntelliJ in the code editor as a graduate thesis project. So as another argument for Eclipse, such efforts as that and PyDev could be combined and utalized in your Zope plugin.

nEO (September 12, 2005 03:44)

I think zope3 need a IDE

Create Iterface, Adapters, Unitest, this is sth. looks like UML tools. And Widgets Scheme Can also be designed like some visual tools.

ZCML need a IDE editor/generator.

And I prefer Eclipse, Because it's mature and powerful

Here is a example for use Eclipse developing J2EE web application http://www.myeclipseide.com/ http://www.myeclipseide.com/images/tutorials/webprojectdemo.htm And more demo in here: http://www.myeclipseide.com/ContentExpress-display-ceid-16.html

J.F. (September 29, 2005 11:45)

I'd vote for eclipse hands down ... The frame work is mature and full of already useful extensions!

I use it for Zope development already, using a combination of PyDev, Subclipse, and the Web Standards Tools and related dependencies.

I can edit Python files intuitively, as well as HTML/XML/CSS!

Some extra functionality like wizards for new objects or templates would be nice extra functionality, or things like SQL Methods?

m.e. (November 04, 2005 04:08)

Zope3 + XUL for rich GUI ?

m.e. (November 04, 2005 04:09)

Zope3 + XUL for rich GUI ?

Sathya Rangaswamy (April 11, 2006 06:14)

This is an interesting conversation. We use homesite to design html and then bring it up in emacs to zptify it. This makes it hard to grow the mindshare of zope plone in India. For example we are implementing a document management system for a major Indian government entity in zope and plone, the biggest challange has been to train their personnel and get their interest. The problem we face is converting the netbeans and visual studio experienced developers who are used to that extensive automation.The development environment for zope plone is a hassle for people who come from such strong IDE based development environments. Meanwhile the other open source based app servers (read JBoss and the like) are making huge inroads into this market as they face less resistance.

Zope 3 and file system based development is a good start. Look forward to better days !

No further comments may be added.

Doctrine and Covenants 10:5 (Click below to fill in the blanks.)
Your browser is not able to display the scripture fill-in program. To see it, enable Javascript or use Mozilla 1.0 or better.

Church: lds scriptures provident games pearls kzion shiblon film chancellor gateway cumorah byutv happiness nephi
Zope: freezope org com zen labs newbies zettai warnes
Python: home pyzine daily icanprogram
Genealogy: cyndi
Weblogs: jeffrey paul jon joel another-shane guido barry jeremy windley chrism zac
News: quakes lwn dc weather deseret zeitgeist softwarelivre
Zaurus: software developer
Tech: tango spintronics thin
Semantic: aaron sean
Reference: css rdf html4 javascript geckodom iecss emacs phrases acronyms
Reverse: advogato slashdot
Misc: gimp-savvy directory soda jokes shouldexist pdphoto