Kate

n/a

Right now, in Kate's gitorious repository we, or rather mainly Christoph, is rewriting the heart of Kate: The text buffer. In this blog I'll explain how the text buffer works in detail. After reading, you'll be able to understand the code. And be warned: Probably you will be desperately amazed about its awesomeness! So this is a must read :^) The following content is based on the notes for design ideas for the new text buffer.

Storage. Kate Part internally stores the content of a text document as a QVector of text lines. Each text line holds the text in a QString object as well as additional data like color attributes for highlighting. Inserting and removing text lines implies inserting or removing items in the QVector of the text lines. This is can get slow when there are thousands of text lines in a document, because QVector has to move the memory every time. Thus, to keep text operations fast, Kate splits all text lines into several text blocks: Each text block contains a certain amount of lines (e.g. 256). The expense of memory movement is then always limited. When a text block grows too much, it is automatically split. When the amount of lines in a text block shrinks, it is merged with the previous text block. In other words, Kate's text buffer automatically balances the text blocks. The basic idea of Kate's text buffer, the text blocks and text lines looks like this:

Text Cursors and Ranges. Text manipulation always takes place at certain positions in a text. Such a position is called a text cursor. Each text cursor is defined by a line and a column. Further, to specify e.g. text selections, we need text ranges. A text range has a start cursor and an end cursor. For instance, if you have two views of the same document, you want the cursor in the 2nd view to keep its correct position in the text if you insert text in the 1st view. Thus, text cursors have to be kind of intelligent. Whenever text changes, each text cursor needs to be moved. As a text range just consists of two text cursors, we will focus only on text cursors for now. The question is how to implement this efficiently? If we store all text cursors in a list in the text buffer, we have to iterate over all text cursors on every editing operation. This obviously does not scale for thousands of text cursors (KDevelop for instance uses thousands of text cursors for the highlighting). The solution is the same as with the text content itself: Let's just put a list of all text cursors in a text block to the text block itself. During text editing, we only have to adapt all text cursors of a single text block instead of all text cursors in a document. This looks as follows:

Editing Operations. When editing text, you usually have only four different types of text manipulation:

1. insert text at a text cursor inside one line
   
2. remove text at a text cursor inside one line
   
3. wrap line at a text cursor (e.g. by hitting <return/enter>)
4. unwrap line (e.g. by hitting <backspace> at the beginning of a line)

Those editing primitives are implemented in the Kate::TextBuffer and Kate::TextBlock and take care of balancing the text blocks. For each one the set of text cursors in a text block has to be adapted. Of course, corner cases need to be taken into account: For instance, unwrapping the first line in a text block means that cursors in the first line need to be moved to the previous text block. All editing primitives emit a signal, such that additional layers later can track what happens. For instance, the undo/redo system needs to know all editing operations. Or on top of that we could implement vim like swap file support, i.e., track all changes from the beginning, and if Kate crashes, replay all the editing primitives on the original files.

Transactions. A transaction consists of several of the four editing operations. For instance, if you select text and then move it to another location in the document with drag & drop you want this to be grouped together to a single text operation on undo/redo. (The text operations in this case are unwrap lines and remove text, then insert text and wrap line). To be able to specify which parts of text operations belong together, the text buffer provides two functions: startEditing() starts a transaction and finishEditing() closes it. Those functions use reference counting, so you can call startEditing() multiple times and only the last finishEditing() completes a transaction. Again, signals are emitted e.g. in finishEditing() such that other layers (undo/redo system) are notified about this.

Revisions. As an easy way to check whether a document changed you can get the current revision of the text buffer. The revision is simply an int64 starting with 0 after loading a document. The revision is incremented in every of the 4 editing primitives. This way you don't have to listen to multiple signals like textInserted() and textRemoved(). This could be useful for e.g. KDevelop's background parser: Parse a file. When done, check whether the revision is the same. If yes, parsing was successful. If not, parse again. This is how QtCreator does it. Easy and straight forward.

Unit Tests. The text buffer, text block, text line, text cursors are all designed such that unit tests can be written. Hence, each of the above implementation details will be covered by unit tests.

Further ideas. As mentioned previously, the design of the text buffer leaves room for further features. For now, let's face two ideas:

• Vim like swap files: If Kate or the application using Kate Part crashes, the unsaved data in the text document is lost. Vim has a backup mechanism called 'swap files' to recover from crashes to avoid data loss since years, and it works as follows: Each of the 4 editing primitives explained above are streamed into a separate file. When you save a file successfully, the swap file is cleared. If a crash happens, vim loads the original text document and then replays all editing operations in the swap file. This way the current unsaved editing state is recovered; no data is lost. Since the text buffer emits all necessary signals, implementing this feature is kind of trivial. Any takers?
  
• Right now, the highlighting works in an extra class, accessing the text buffer's text lines and setting the text attributes such as the color. As idea, we could also derive a class Kate::HighlightingBuffer from Kate::TextBuffer and do the additional modifications in the derived class instead of an extra class. We'll have to see whether that makes sense.

Please Contribute.

• join #kate on irc://irc.kde.org
• checkout the code base, clone it, create branches, and merge back!
• other kate internals: undo/redo system (little bit outdated, but all in all still valid)
  

We are about to move the applications Kate and KWrite as well as the libraries KTextEditor and Kate Part to gitorious. Christoph is working on the migration right now in order to keep the development history. Things look good so far, so the migration is soon finished.
We have discussed a bit about the migration to gitorious on the Kate Developer Meeting and Christoph came up with this mainly because building only KTextEditor, Kate Part, KWrite and Kate is much faster and easier compared to building the KDE modules kdesupport, kdelibs, kdepimlibs, kdebase, kdesdk.
I myself remember the time where I started KDE development, and it took more than two weeks to have a first successful build of KDE. You have to learn so many things at once, like revision control, lots of so far unknown software, and what not. Talking to other developers verifies this. In other words: Getting into KDE development is not easy and straight forward.
Moving to gitorious removes this barrier for Kate development: You just checkout the Kate repository and that's all you need. It would be nice if you join Kate development and contribute patches :)
What does that mean for Kate in KDE? Nothing changes. We will merge the changes in Gitorious back to the main KDE development line and vice versa.

Since several years, we have lots of small developer meetings in the KDE project, gratefully supported by the KDE e.V. There, developers of a certain project (e.g. KMail/kdepim, Plasma, you name it) meet to discuss further plans and to push the project further. From experience we can tell that those meetings are really beneficial in several ways:

• Social aspect: You get to know the other developers involved in the project in real life, which is a great motivation factor. This also happens at KDE's annual conference Akademy, although there are a lot more people.
  
• Productivity: Since you are sitting next to each other discussions about how to do what are very focused. It's amazing how quickly a project can evolve this way. (I haven't seen such focused work in companies, yet. I guess the variance in the knowledge of people involved is higher. And the motivation is usually very different).
  
• Knowledge Transfer: Since participants are experts in different areas, discussions lead to knowledge transfer. This is essential, as sometimes developers have very few free time to contributes to a project. Spreading the knowledge helps a lot to keep the project alive.
  
• Steady Contributions: We are always open for new contributors. Just send a patch, get commit access and join development. Experience shows that participants of a developer meeting usually contribute for years to come.

Enough said, here is what happened the last three days in Kate:

• memory optimization: remove unused edit history items
  
• fix crash in doHighlight due to wrong initialization
  
• don't squeeze captions in the window title
  
• fix crash when using katepart in e.g. konqueror
  
• kate vi mode improvements
  
• fix: invalid js commands lead to a crash
  
• share code snippets between kate and kdevelop
  
• more syntax highlighting files for asm dialects
  
• replace: show number of replacements in a passive popup
  
• new command line command: unwrap
  
• fix crash when switching sessions with multiple main windows
  
• fix incorrect global replace %s///g
  
• fix crash in smart range group handling
  
• turn search option "highlight all" into the action "find all"
  
• replace old text snippet plugin with new one
  
• optimize away nops for text editing
  
• fixes in the template handler code
  
• remove pythong browser plugin, since the symbol viewer plugin can do the same and much more
  
• remove unmaintained html tools plugin, since it was never ported
  
• optimization for bracket matching
  
• new text snippet plugin: context aware snippet completion
  
• fix filtering in the file browser plugin
  
• always start search&replace from cursor
  
• developer doumentation for smart range usage
  
• cstyle indenter: fix indentation of multi-line comments
  
• update of syntax highlighting for MySQL
  
• performance optimization in the highlighting system: bug, bug
  
• fix crash in removeTrailingSpaces, if undo/redo is active
  
• new interface for getting the mode at a given cursor
  
• search & replace: lots of code cleanups
  
• basic regression tests for search, scripting, and more...
  
• scripting: add support for quoting in command line scripts
  
• Kate crashes because of spell checking
  
• fix regression: with auto brackets + selection
  
• find-in-files: set filter correctly when opening the search
  
• fix: make reload-scripts work with new functions
  
• share a KCompletion object between all KateCmdLineEdits
  

There are even more changes I left out. Most of those changes will be in KDE 4.4.1. If you want to help, join #kate in irc.kde.org!

We are 10 people here at the developer meeting in Berlin. Kate, KDevelop as well as Okteta see a lot of commits. I'll mainly talk about what's happening in the holy Kate land, though :-)
Yesterday I closed a bug requesting an "unwrap" feature in Kate that works like "Tools > Join Lines" but maintains paragraph separation, i.e., empty lines are not removed. This feature is implemented now in javascript. Further infos:

• the bug report
• the javascript implementation

To run the script simply switch to the command line (F7) and write "unwrap". If you have further ideas about useful scripts, don't hesitate to start hacking right away, see also

• how to extend Kate with scripting

Fixes with regard to the scripting support in the last days are

• make reload-scripts work properly
• fix command line completion for new scripts without restarting kate
  
• support quoting in arguments for scripts
• fix crash for invalid commands in scripts

Those fixes will be in KDE 4.4.1. More to come in other blog entries :-)

In KDE3 Kate had a plugin called "XML Completion Plugin". It was able to list XML elements, attributes and attribute values and entities in a completion popup menu depending on the currently active DTD. For instance, it was able to complete all your KDE/docbook tags and, thus, was one of the best tools for writing KDE documentation.

Unfortunately, this plugin has not been ported to KDE4, yet. So this is a request for developers: Please someone pick it up and make it available for Kate in KDE4.

The location in svn is: trunk/KDE/kdesdk/kate/plugins/xmltools
Just enable this folder in the kate/plugins/CMakeLists.txt file and start porting it. The code completion interface changed quite a lot, but there are other plugins and good api documentation (KTextEditor interfaces, Kate Application interfaces) where you can have a look at how things work.

Having a working XML Completion plugin for KDE4.5 would be awesome.

Det er længe siden jeg har skrevet her, af forskellige årsager, så her følger lige en opdatering.

Alweb.dk har været delvis nede fordi en tabel i databasen var brudt sammen, det var heldigvil let nok at reparere. (Man kunne ikke logge ind, eller se andre sider end forsiden. Det har ikke holdt mig tilbage, jeg vidste det simpelthen ikke.)

Vi har fået ny leder på mit arbejde, og det er trættende — ikke fordi der er noget i vejen med hende, men det giver jo uro i arbejdsformerne. Forhåbentlig kommer der gode ting ud af det, meget peger den vej.

Og så har jeg forresten fået lidt flere timer, jeg arbejder 35 timer om ugen for tiden. Det giver en lille smule mere i løn, og en del mindre tid.

Jeg har skiftet styresystem, jeg blev træt af KUbuntu fordi de bestemte at det ikke var nødvendigt at understøtte KDE 3 i den nye version, og det er nødvendigt for mig. Jeg kunne have beholdt den gamle version, men så løber man ind i problemer fordi KUbuntu ikke laver opdateringer af programmerne udover sikkerheds-. Altså, farvel og tak til KUbuntu som jeg ellers har været glad for at bruge i ret lang tid.

Jeg forsøgte med Debian, men jeg blev sur på det fordi valg af ‘desktop’ betød installation af en masse programmer jeg ikke vil bruge. Hvis jeg skal vende tilbage til Debian engang, vil jeg starte med den mindst mulige installation.

Så forsøgte jeg med OpenSUSE, men det fungerede simpelthen ikke. For at få de programmer jeg skal bruge skulle jeg installere fra forskellige kilder, og deres softwaremanager havde (jeg håber for dem at det er løst) en fejl så den ikke valgte den korrekte version af programmerne til installation. Og selv da jeg endelig fik installeret det jeg ville, var der hele tiden sammenbrud og problemer. Understøttelsen af KDE 3 var der, men problematisk, jeg kunne f.eks. ikke installere hjælp for kde3 programmer.

Nu har jeg installeret Arch Linux, og det er godt synes jeg. Der installeres en minimal version, og man installerer så efterfølgende hvad man vil. Softwarehåndteringen er glimrende, let at forstå og bruge. I Arch Linux skal man konfigurere al ting selv, det er lidt omstændeligt, men til gengæld har de en virkelig god dokumentation på deres wiki. Der er nogle ting der ikke fungerer endnu, suspend er en og knetworkmanager en anden (det starter, men kan ikke forbinde, pga konfigurationsfejl tror jeg). En af de store fordele ved Arch Linux er at de har løbende udgivelse af opdateret software. Dermed kan man få nye versioner når de kommer, men man kan selvfølgelig beholde kritiske dele af systemet som man ønsker.

Jeg har bygget en del af KDEs udviklingsversion forleden, så jeg kan komme igang med at hjælpe lidt med Kate igen. Jeg har allerede sendt det første simple bidrag :-)

Og så er jeg igang med at hjælpe lidt med de danske oversættelser af KDE, mest som korrektur-læser, men det er jo også en en vigtig opgave. Det er spændende og giver nogle sjove oplevelser ind i mellem når oversætteren ikke aner hvad teksten handler om, eller har for travlt til at opdage nogle af de sjove ord, forleden stødte jeg på “tidsintervaldørstopper” ;)

Idag sidder jeg og tænker på økonomi — hvad skal der til at male/vedligeholde Kate, har jeg råd til at købe et kamera? Jeg kigger i den blå avis efter et godt digitalt spejlreflekskamera, noget i retning af et Canon EOS 30D, som kan fås til en ret god pris. Det kunne være sjovt, men jeg skal altså lige tænke mig om…

read more