Monday, September 13, 2010

Whyfor CodeAssistor?

Time for some introspection. Why did I create the CodeAssistor? Isn't there lots of text editors? Why Open Source? What have I learned about Open Source so far?

Initial Inspiration (the itch!)

It was late 2009, and I was interested in looking at the Android RIL code. I work for research in motion on the QCT radio team and was interested in how Android did their system determination stuff for multimode devices. I wanted to look at some of their source and compare against some of the code that I work on everyday. I downloaded the code and started using "xed" to view the source. Xed was my only editor that I had on my Mac at the time.

(BTW, all the really interesting code is hidden from you in Android, at the time Qualcomm had not open sourced the libraries underneath the basic high level stuff... I seriously doubt that they've open sourced the interesting stuff even now... a word of warning for those of you who hold up Android as a pinnacle of freedom and open sourced software... I wasn't able to glean any useful information)

Xed is the editor component for the xCode IDE created by Apple, and although xCode has some terrific components xed started to annoy me for two reasons:
  1. when you run xed, you start up the whole xCode IDE - This takes time to load, sometimes 5-8 seconds. Since Im not working on the code itself, I am simply reading it I didn't want a full IDE, and I wanted some snappy performance, so I could open/close source files quickly
  2. xed uses a different look and feel from all other editors I've used - It's rather hard to describe the look/feel of xed, for those of you who have used xed, you'll know what I mean. While other people may like how xed runs, I wanted something more traditional.
So I downloaded some other editors. I tried TextWrangler first. With TextWrangler I found some bugs relating to opening files from a state where the application is not running... I'd have to open the file twice! Very annoying. Its not open source, so I felt no compultion to submit bug reports (if you are Cathedral style, you're bugs are your own). I then tried Mac ports for VI and eMacs... but I dont really know how to use those editors (I can survive on vi, but I dont want to... I am completely lost in emacs), so they were out. Then I tried jEdit. jEdit was pretty good, but it has the same problem every other java program has... its slow to start.

All these editors that I tried were also very feature rich. Overly feature rich, I simply didn't use most of the features, and the menubars were too full of things. I got lost. Not only this, but the feature-riches bloated the overall size of the executables. Some of them are 30Megs or more in size!

In parallel to this, I use an amazing source viewer at work called SourceInsight. SourceInsight is ABSOLUTELY the best code reading software on the market. This software is like a graphical cscope, but more robust and reliable. Unfortunately SourceInsight has a few major problems:
  1. It's Windows Only - no ways around this... I'd have to run a virtual machine to use it
  2. It's extremely expensive - something like $250 a license
  3. It's not maintained anymore - features like code folding are not supported.
  4. It's an MDI application, yet all editor windows are bound inside a master window... thus it doesn't let you take advantage of multiple monitors
  5. I dont think it supports Objective-C
So I got to thinking... what if I created my own editor. If I created my own editor I could
  1. add a source analyzer engine similar to cscope (like sourceinsight) for easy code jumping
  2. make it small
  3. make it run quickly
  4. make it simple to use
  5. make it support all the languages I care about (Objective-C isn't well supported in editors)
I've succeeded in items 2-5 (#1 is a doozy, its going to take a really long time to do that)

Beginning to Code (Enter the Scintilla!)

It is quickly apparent that writing a whole editor from scratch is a hugely difficult task. Luckily Scintilla handles all the editing tasks for me (folding/colourizing/etc). Unfortunately I found two problems with scintilla
  1. Scintilla is a second class citizen on Mac
  2. I dont like how Scintilla's folding
Regarding #1. Scintilla has been ported to MacOSX in two ways. A Carbon based port, and a Cocoa based port. I actually built a small editor (no frills like saving or anything... but you could type in it) in Carbon, and I found that programming in Carbon is rather disgusting. Its way overly complex and requires a huge learning curve. Add to carbons detriments the fact that programs written in it will never be 64 bit, and I abandoned this and moved to Cocoa.

Cocoa had it's own problems. The first is that at the time, it was only buildable via xCode (isn't moving away from xCode one of the major reasons Im making an editor?). There is a simple solution to this... learn about the xCode build process, and port the building of Cocoa-Scintilla to makefiles. This wasn't an easy task, but I got it done. I submitted my fixes into the official tree of scintilla, so I got a warm feeling for contributing back to the community; I also learned a lot about makefiles which has become useful for my dayjob. Cocoa's second problem is that it uses ATSUI (hehehe Dave are you reading this? Remember that the stuff I was looking at for DPS was ATSUI stuff?).

While ATSUI is great, its carbon based and will hold back porting to 64bit. The new Mac Text stuff is now CoreText. I haven't yet begun to port Scintilla to use CoreText, but its on my future TODO list (its important to simply get the editor working with what exists right now).

Regarding #2. I figured out the way Scintilla folds and did some kludging to make it work the way I want it to. I haven't proposed this code back to the scintilla because what I've written isn't production level code... its a dirty hack. I've released my patches for anyone to see if they are interested in how I am doing it. Sometime soon I plan on cleaning things up, doing it properly and proposing the code back to the Scintilla team.

Why Open Source?

Like I described with TextWrangler, if you run a Cathedral style project, you're bugs are your own. However with open sourced bazaar style, bugs belong to the community. As a single developer I simply dont have time to fully test my code (did I just admit that? did I just say I was a lazy programmer! yes indeed I did. If you've got a problem with that YOU CAN HELP ME OUT). I cant run through every usecase on every platform, but I CAN run through all the use cases I care about (I use CodeAssistor at work (windows port) and at home (mac port)... I'll find most of the normal use use-cases via regular usage... you'll note that I've left out linux port for regular usage HINT).

Using the bazaar style of project management, any bugs that I do not find via regular usage can be found by my users. As a bonus, my users are all programmers, so if they find bugs I think it more likely that they'll help me via patches since they can examine the code.

As a side note: This is why I chose Launchpad to host my source code. Launchpad makes "drive by contributions" much easier than sourceforge or google code. Go read up on it here.

Why BSD? Complex answer, I think BSD is more free than GPL. The BSD is truly free in that people can use the code to do nearly anything with the code (see my license). GPL restricts people from doing potentially interesting stuff with the code. To say that people wont submit code back to BSD where they would submit back to GPL is silly... if someone else has patches on top of my code they recently got SCREWED because I did some major refactoring; they'll now have to do a lot of work to maintain their patches... its easier and better for them to submit code to be incorporated into the trunk of the code.

Short Answer: GPL is way to legalize for me to truly comprehend it... best not to use licenses you don't really understand. BSD is simple and does what I want it to do.

What Have I Learned from Open Sourcing CodeAssistor?

The most important thing I've learned is that it doesn't really matter if you are open sourced or not via the standpoint of bugs until you can create a community. I've had nearly 550 downloads of version 0.0.5 and not a single bugreport (I found a major one while I was late into 0.0.6). "If you build it they will come" is not the way of open source... creating a community is something I'll have to work on, and it's going to be a difficult task.

As a plug, 0.0.6 is the most interesting time to jump on board the codeassistor community. Its now running rather well on the three major platforms, but there are lots of features still to do (the most interesting features at that!).

Another thing I've learned is that there are lots of shareware sites which will grab your stuff, claim it as "shareware" , and use it to promote their sites. Some of them technically they violate my license because they dont include my license demand #3 (see the licence). It's not worth it to hire a lawyer, and Im kind of thankful that they are trying to get the software into the hands of more people. Some of them like Softpedia were very nice in sending me an email and asking if it was ok to include the software, and making sure that my name is on the site as the author; in cases like this these sites are Win-Win.


I'm really looking for some contributors now. CodeAssistor is on the cusp of doing some really interesting stuff soon; CoreText implementation in Scintilla, cleaning & submitting the fold hack back to the Scintilla community, source code analysis engine. At +4000 lines (66,000 for scintilla), CodeAssistor is small enough to be quickly understandable, yet big enough to be complex and intellectually stimulating. It's a lead in towards Scintilla, winner of the Project of the Month for July from Sourceforge, and one of the truly inspirational open source libraries with an active community It uses three different languages (plain C, C++, and Objective-C), and runs on three major platforms, so it's an interesting learning opportunity towards cross-language cross-platform coding.

With help, I think that CodeAssistor can be a meaningful and widely used editor.

No comments: