Revision as of 17:14, 6 November 2015

The Comprehensive Guide to SWT Development for Linux/Gtk

This is a work in progress. Certain sections may be missing, or incomplete.

About SWT

SWT is the layer between PlatformUI and the underlying Gtk2 or Gtk3 of the system. It can be launched to use Gtk2 or Gtk3, but cannot use them both at the same time. Communication from Java (SWT layer) to C (native Gtk layer) occurs through calls defined in OS.java. The image below describes this hierarchy better:

You will learn:

• How to configure your machine (Eclipse/Git) to contribute SWT patches
• Info on building .SO images to run snippets with newest SWT Master
• Info on Gtk Versions, compiling various Gtk Versions
• How to add your own gtk_ methods to OS.java
• Tips on SWT Development, how to navigate the code base
• Understanding the underlying glue (swtFixed) between SWT/Java and Gtk/C
• Learn how to compile various Gtk versions for testing with SWT
• Learn how to make a Gtk application in Eclipse & how to debug it
• Learn to debug Gtk itself
• Learn how to debug the native Gtk (C) part of a running SWT (Java) application
• Learn how to debug the swtFixed custom C code of a running SWT (Java) application

Upon completion of this document, hopefully you will know SWT-fu (as in kung-fu).

Knowledge Pre-requisites

• Solid Java knowledge: from class inheritence to multi-threading
• Some C experience: know thy pointers and make files
• Gtk background highly recommended but can be learned
• Eclipse: knowing how to use Eclipse is quite essential. But you don't have to know about JFace/PlatformUI/RCP development internals
• Git: for example you should know the difference between merge and rebase. But all git-bits can be learned

Communication and support

SWT Community

If you have questions, you should post them to:

 platform-swt-inbox@eclipse.org

It's also a good idea to sign up for the SWT mailing list. There are #swt-gtk and #eclipse-dev channels on Freenode (IRC) as well.

Following SWT bugs in Bugzilla

Whenever you work on a project, you should consider following the default assignee of the project in Bugzilla. This way you find out about new issues and what issues get worked on. To do so, go to bugzilla https://bugs.eclipse.org/ -> Preferences -> Email preferences, and add the email below to your watch list:

 platform-swt-inbox@eclipse.org

SWT Development Environment Setup

There is an important distinction when running SWT: using the pre-compiled .JAR file, or using the source code in your local Git repository.

.JAR

You can download the SWT .JAR file, and add it as a library in your project: https://www.eclipse.org/swt/. However, when you make changes to the SWT source code, these changes will not be visible when running SWT. This is only useful if you develop 'using' SWT, but not SWT itself.

Source code

To develop SWT itself, being able to run SWT using the changes made to the source code is important. The process to do is lengthy, but valuable.

Add the SWT project to the build path

In order to run snippets with the SWT source code, you will need to add the SWT project to the snippets' project build path. To do this:

• Right click the SWT snippets project in the Package Exlporer
• Select "Properties"
• Select "Java Build Path" on the right hand side
• Select the "Projects" tab and click "Add..."
• Select the SWT project and click "OK"

Install the SWT Developer Tools Eclipse Plugin

SWT developer tools automatically builds the custom C code that SWT uses to work with Gtk. Without the tools, you cannot develop SWT. Install the latest from the Eclipse update site: https://www.eclipse.org/swt/updatesite.php

Remember to re-install these tools if you re-install Eclipse.

Eclipse git (Egit)

Eclipse git is used to compare SWT files against older versions. Not strictly necessary, but highly recommended as it's sometimes easier to use Git through Eclipse than it is through the command line. If you are on Fedora, run:

 sudo dnf install eclipse-git

SWT source code and binary repositories

Check out the repositories holding the SWT sources and binaries. I usually do this from inside Eclipse, but you can also clone things from the command line. The URI's for the repositories are:

 git://git.eclipse.org/gitroot/platform/eclipse.platform.swt.git
 git://git.eclipse.org/gitroot/platform/eclipse.platform.swt.binaries.git

Modify the .classpath files

At first you might get many many errors. You first have to tell the project, that you are on Linux/Gtk for things to compile/run properly. Specifically for Gtk, you must do the following:

• Open the 'Navigator' view (not package explorer)
• Under 'org.eclipse.swt' look for the .classpath files
• Rename .classpath_gtk to .classpath
• Clean up projects (in Eclipse, Project -> Clean)
• Now run a test snippet or ControlExample.java, it should work without the compilation errors

Configure git for review

To contribute to SWT, you need to modify your Git config to be able to push to Gerrit, which is the Eclipse Project's review system. Open your Git root, navigate to the SWT repository, and open the Git config file with a text editor . On my system it looks like this:

 ~/git/eclipse.platform.swt/.git/config

Add review branch: (adjust for your own user name instead of 'lufimtsev'):

 [remote "review"]
   url = ssh://lufimtsev@git.eclipse.org:29418/platform/eclipse.platform.swt.git
   push = HEAD:refs/for/master

Under the existing master, add 'rebase = true':

 [branch "master"]
   rebase = true

Add your name and email for signing off commits:

 [user]
   name = Lev Ufimtsev
   email = lufimtse@redhat.com

You can now push your changes to the review branch (Gerrit) for review:

 git push review

Enable 32/64 bit checking by SWT Tools

The Java source code has to work on both 32 and 64 bit machines. On 32 bit machines, calls to Gtk that use long in the parameter will not compile. All such longs should have an /*int*/ annotation after it. For example:

 //This will cause a 32 bit build to fail:
 void gtk_css_provider_load_from_css (long context, String css)  { .. }
  //                                      ^ missing /*int*/
 
 //Every 'long' should followed by a '/*int*/' like so:
 void gtk_css_provider_load_from_css (long /*int*/ context, String css)

To avoid missing these things by accident, SWT Tools can automatically check these things and mark them as errors. This functionality must first be enabled though. By now you should have SWT Developer tools already installed. To enable checking automatically:

• Right click on the 'org.eclipse.swt' project.
• From the drop down menu, enable 'SWT Tools -> Report 32/64 bit problems'
• Now if you don't include the /*int*/ a warning will be thrown in your the problem view

References

https://www.eclipse.org/swt/fixbugs.php

Gtk .SO bindings

About

You checked out two SWT repos: SWT and SWT binaries. The SWT binaries contain ready-build SO files to be used with built with the SWT source code. Sometimes you need to add native Gtk functions to OS.java, allowing you to use those new functions in the SWT (Java) source code. After any changes to OS.java, you need to rebuild the Gtk .SO bindings. Otherwise SWT will complain about "unsatisfied link errors" between the SWT source code and the binaries.

Prerequisites

• SWT Tools
• Gtk libraries:

 #non-fedora, install these packages:
 gtk3-devel gtk2-devel libXtst-devel mesa-libGLU-devel libXt-devel
 //you may also need "X Software Development".
 
 #On Fedora, you can auto install all libs needed for Gtk2 building via:
 sudo dnf builddep gtk2   
 sudo dnf builddep gtk3   
 sudo dnf groupinstall "X Software Development"

Building

This process can (and is) automated into a script, but knowing what is going on in the background is also important.

The below assumes that you checked things out into:

 ~/git/

• Clean the org.eclipse.swt project in Eclipse (Project -> Clean)
• Inside Eclipse, run build.xml from org.eclipse.swt.gtk.linux.x86_64/build.xml and make sure "build.jars" and "build_libaries" are selected
• You will get some error messages, this is fine. Proceed with the next step. The error message will look similar to this:

 /home/lufimtse/git/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml:918: The following error occurred while executing this line:
 /home/lufimtse/git/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml:890: javax.script.ScriptException: ReferenceError: "importClass" is not defined in <eval> at line number 3

• Open terminal, go to /bin/library inside your SWT repository:

 cd ~/git/eclipse.platform.swt/bundles/org.eclipse.swt/bin/library/

• Set correct export variables: (otherwise you get "fatal error: jni.h: No such file or directory")

 export JAVA_HOME=/usr/lib/jvm/java/
 export GTK_VERSION=2.0

• Rebuild the wrapper:

 sh ./build.sh

You may get some missing library errors, like:

 fatal error: X11/extensions/XTest.h: No such file or directory
 #include <X11/extensions/XTest.h>

In this case you will need to look at the library in question and install it.

• Copy the new .SO files across:

 cp -v ~/git/eclipse.platform.swt/bundles/org.eclipse.swt/bin/library/*.so ~/git/eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/

• Building for Gtk3: repeat the previous steps except export version 3 for Gtk:

 export GTK_VERSION=3.0

This process is time consuming. To simplify it, download this build script: https://github.com/LeoUfimtsev/ldts/blob/master/pathscripts/swtjnibuild

To run it, simply do a clean (Project -> Clean) of the SWT project and then execute the script. For Gtk3, no arguments to the script are needed. To build the Gtk2 bindings, run the script with the "-2" option.

References

https://www.eclipse.org/swt/jnigen.php

The SWT codebase

Learning SWT

To learn about SWT, try out a bunch of these snippets: https://www.eclipse.org/swt/snippets/

Note, these are available in the SWT repo, search for "Snippet1.java". The org.eclipse.swt.examples project contains lots of examples to try out. There are also many many SWT tutorials online. For example, http://zetcode.com/gui/javaswt/ comes to mind.

You should learn at least:

• Set of basic widgets (Button/Label/Table/Tree etc..)
• Layouts (Absolute [i.e no layout]/Grid/Row/Column/FormAttach)
• Attaching listeners

General Widget Hierarchy

Widget is the main Widget. Everything else extends Widget. The most interesting classes are Widget, Control and Composite. Most widgets fork off of these.

As such, it is useful to be aware of fields/methods in parent classes and which methods get overriden by children. In Eclipse, you can Ctrl+click on a method to see it's super implementation, or derived implementations (this is very useful).

The illustration below shows the widget hierarchy:

About GtkHandles

Gtk Handles are basically Gtk pointers to the widgets.

When you first look at a widget, you generally first look at the handles that are defined in the widget. For example, each SWT Widget is often made up of several GtkWidgets or has pointers to GtkWidgets. In general createHandle() is a good place to start when you first look at a widget. Also inspect *Handle*() methods of the current widget and the widgets above in the hierarchy: parentHandle(), fontHandle(), childHandle(), etc. They all contain subtleties which may useful.

Handle declarations are inherited. But they are allocated and assigned at a specific widget level. For example Widget declares handle, Control declares fixedHandle but they never allocate them. These will be allocated further down the widget tree in individual widgets themselves.

For example, here is an incomplete diagram of some widgets and their handles. You may observe that widget has handle defined, and Combo has buttonHandle defined:

OS.java

All Java calls are eventually translated to GTK to do the actual drawing.

The bridge between Java and GTK is in OS.java. OS.java is a file that we write ourselves (not generated). It contains many bindings to GTK functions, but not all of them. Sometimes we have to add method signatures in there manually for new functions.

In my case, it is located here in the package path:

 /org.eclipse.swt/Eclipse SWT PI/gtk/org/eclipse/swt/internal/gtk/OS.java

Method Bindings

These are typically wrapped in a lock:

 /** @param widget cast=(GtkWidget *) */
 public static final native void _gtk_drag_dest_unset(long /*int*/ widget);
 public static final void gtk_drag_dest_unset(long /*int*/ widget) {
         lock.lock();
         try {
                 _gtk_drag_dest_unset(widget);
         } finally {
                 lock.unlock();
         }
 }

Constants and ENUMS

C constants and C Enums are delcared as plain ints. like so:

 public static final int GTK_SCROLL_STEP_UP = 6;

Note, if you need to add a C enum to OS.java, C ENUMS (Enumirations) begin at 0. Do note, that the Javadoc is parsed by the JNI parser. So be careful about what you put in there.

Static Strings translated to C-bytes

GTK uses some static strings in things like registering event handlers. We translate Java strings into C strings like so:

 public static final byte[] key_press_event = ascii("key-press-event");

GTK version

A lot of SWT code is wrapped around the if-checks that make sure code runs only on certain GTK versions. The version code is defined in OS.java like so:

 public static final boolean GTK3 = GTK_VERSION >= VERSION(3, 0, 0);

OS.Java compilation

SWT tools compiles OS.java down to a set of files like os.c. Be attentive, there are two sets of library files like os_custom.c. This is because one set is the source, the second is copied over during compilation.

Source folder is here (this is where you should make changes):

 ~/git/eclipse.platform.swt/bundles/org.eclipse.swt/Eclipse SWT PI/gtk/library/os_custom.c

Destination/binary/compiled folder is here:

 ~/git/eclipse.platform.swt/bundles/org.eclipse.swt/bin/library/os_custom.c

It is important to understand the difference because later you will be making changes in the source folder, but link the debugger to the source code in the destination/bin folder. Do note, inside the source folder, some of the files we adjust by hand and some of these are generated. The content of ../library/.. is as following:

os.c

 This file is automatically generated by SWT Tools. It contains native bindings.

os.h

 This file we adjust manually. It contains special signatures.

os_stats.*

 These are auto-generated.

os_custom.h

 This file we adjust manually. We add new function signatures here when we add a new method to OS.java.

os_custom.c

 This file we adjust manually. This contains our custom code, such as the SWTFixed container.

When SWT Tools re-generates os.h os.c etc., they may appear in your git staging area. When submitting a patch, include these in the submission.

SWT Code style notes

Javadocs

No platform dependent info: no technical details that are platform-specific go into Javadocs. (e.g no GTK specific items). putting them in regular non-Javadoc (green) comments is OK.

Method naming

GTK functions: any GTK-specific functionality should be gtk_function_name. Even if it doesn't match one-to-one to a gtk function. (do try to avoid conflicts thou).

Any other naming should be general enough to be translated to other platforms.

Method access modifiers

Methods should in general not have modifiers unless private/public.

 void myMethod()...

Avoid 'protected' unless you're sub-classing things.

SWT Bug naming conventions

Cheese

This refers to garbled text or messed up pixels.

DnD

Usually used for Drag and Drop.

Win32/Cocoa/Gtk

These you will see as prefixes in bug-titles. They stand for the different platforms that SWT uses for the Eclipse UI. On Windows it's Win32, on Mac/OSX it's Cocoa, and on Linux it's GTK.

SWT Fixed Container

SWTFixed is custom C code that we include in SWT. It is a container that allows us to place SWT Widgets with absolute positioning so that subsequent widgets are drawn beneath each other. (In GtkFixed container, subsequent widgets are drawn on top of previous once). This is achieved by using the gtk_widget_show_unraised() method, (show widget, but do not raise it up).

As such, the swt_fixed_for_all() method that traverses the code is a bit different than standard for_all() methods.

SWTFixed was introduced in SWT during the GTK2 to GTK3 migration, because the old GtkFixed container was removed in GTK3.

SWTFixed is defined in os_custom.h and os_custom.c. In my case they are here:

 ~/git/eclipse.platform.swt/bundles/org.eclipse.swt/Eclipse SWT PI/gtk/library/os_custom.c

Every widget is based on SWTFixed

If you inspect a widget, in almost every createHandle() method, you will see the creation and allocation of SWTFixed like so:

 fixedHandle = OS.g_object_new (display.gtk_fixed_get_type (), 0);
 OS.gtk_widget_set_has_window (fixedHandle, true);

This means that SWT is creating a new instance of the SWTFixed container. Normally it then assigns something to the handle variable puts the widgets inside it.

Usually in GTK you create a drawing surface (GdkWindow) [note the 'd'] and draw one or several widgets inside it. In SWT, we create a new drawing surface (GdkWindow) for every single widget. This is very ineffective but gives SWT more control about drawing order and permits easier implementation of overlapping widgets. At this point it might be beneficial to understand that a GdkWindow is a drawing surface, where as a GtkWindow [note 't vs d' difference] is a shell with decorations like 'X', '-' and it interacts with the X drawing system. More will be discussed in the GTK sections that follow.

Technical C details

You might want to read a bit on how to implement a custom widget in GTK. Then SWTFixed will be much easier to understand. It's mostly code copied from other generic GTK containers with a few tweaks here and there.

Specifically:

• SWTFixed extends GtkContainer, i.e it inherits all its functions. This can be observed by the SWTFixedPrivate struct, it has a reference to GtkContainer meaning that the GtkContainer widget is its parent
• Supports multiple children because the child object is a list.
• It overrides some function, this can be observed in the init() function, which assigns functions to pointers. I.e.:

 container_class->forall = swt_fixed_forall;

SWT and GTK versions

With SWT, the version of GTK that you're running on can make a big difference in how your snippet will behave. This is especially true for all things CSS related because there are many changes in that area.

It's not just the difference between GTK2 & GTK3, but keep an eye on the differences between:

• GTK3.8: RHEL 7 and Fedora 19
• GTK3.10: CSS theming introduced, lots of things break between GTK3.8 and GTK3.10 because of this
• GTK3.12: nothing super exciting
• GTK3.14: Fedora 21
• GTK3.16: Fedora 22, removal of stock icons
• GTK3.18: Fedora 23, re-skin of File Chooser, gtk_widget_override_* functions become deprecated

Because of the faster pace at which the GTK development cycle works, it is imperative to be able to test on different versions of GTK (within reason). Usually testing 2-3 versions behind is useful for Fedora. Try to test on GTK3.8 as RHEL customers use this and require an especially stable platform. GTK2 is worth testing on to make sure no regressions are introduced (as SWT currently supports GTK2), but we are not fixing any new GTK2 bugs.

To find out which exact versions are currently supported by SWT, inspect GTK's Display.java:

http://git.eclipse.org/c/platform/eclipse.platform.swt.git/tree/bundles/org.eclipse.swt/Eclipse%20SWT/gtk/org/eclipse/swt/widgets/Display.java

And search for code like this:

 /* GTK Version */
 static final int GTK3_MAJOR = 3;
 static final int GTK3_MINOR = 0;
 static final int GTK3_MICRO = 0;
 static final int GTK2_MAJOR = 2;
 static final int GTK2_MINOR = 18;
 static final int GTK2_MICRO = 0;

This means GTK3.0.0 and GTK2.18.0 onward are supported.

Once in a while, you will come across bugs where the version is bumped. Keep an eye on this by following platform-swt-inbox@eclipse.org in the Eclipse Bugzilla. For example:

 https://bugs.eclipse.org/bugs/show_bug.cgi?id=446454