StarExec is written almost exclusively in Java aside from the actual distribution and running of jobs which is mostly done with scripting on the Linux operating system. We have a wide variety of Java classes that represent certain objects in the system such as solvers and benchmarks. We also have Java classes that are responsible for such things as submitting jobs and managing the database. Advanced knowledge of Java is required for this application, specifically thread-safe programming.
Java Server Pages
StarExec utilizes the Java Server Pages platform (JSP) as its user interface (front-end) technology. This allows us to easily present things to the user that exist as Java objects in the core application, as well as process incoming information and take appropriate action with certain Java classes. Another advantage is the ability to utilize an extensive and powerful collection of tools and libraries that Java provides.
Tomcat is the web application container we use which "hosts" the StarExec web application. You can think of Tomcat as a manager that is responsible for redirecting requests to certain webapps (in our case we only have one, but you could have multiple) and provides important functionality like restricting access to content, etc. As a developer you won't have to worry much about Tomcat since it is maintained by CSG. Currently the only time we delve into Tomcat is to change security configurations.
Traditionally in older versions of JSP you had to embed Java into your page which is messy and ugly. Essentially you're combining business logic with layout which is always a bad idea since it makes a simple layout harder to maintain. To get around this problem, Expression Language (EL) was introduced to make it easier to insert dynamic content into a page.
For example, this is how a user's name would be inserted into a JSP the old way:
and this is the simplified way with EL:
Expression Language is very simple and intended for web designers to use so as to avoid learning programming. However it is still almost an entire other language on its own so it may require some reading up on to fully understand.
Java Standard Tag Library
Another problem with the old version of JSP is control flow. It was messy to implement loops and conditionals with Java and mix it with HTML. For this reason, the Java Standard Tag Library (JSTL) was created and designed to work with EL. For example say we wanted to display an html element based on some condition. First at the top of the page we must declared that we intend to use JSTL.
The taglib directive allows us to specify which tag libraries to use. The most standard one is the JSTL core library which is usually prefixed with
c. The if tag is one of many special tags that JSTL provides for us. Another useful JSTL library is the function library.
Here's an example that utilizes a simple function:
On top of the JSTL core and function libraries, you can also create your custom tags that you can specify the behaviour of. We utilize this to provide site templating. For instance we have a template.tag file that specifies the layout of all the common elements of a page (headers, footers, menus, logos, etc.) Then we use that tag in other pages where we can just specify the main content.
For example, here's a custom tag that represents the site's template (we'll call this template.tag):
Now here is how we would use that template in some welcome page:
Oracle Grid Engine
The Oracle Grid Engine is a system used to distribute jobs to worker nodes and manage the queue/workload of the jobs in the system. For us, this means the Grid Engine will distribute solvers and benchmarks to nodes to be solved locally. This takes the responsibility of managing a scheduling and queuing system off our hands.
We will be interfacing with the grid engine through a package Oracle provides called DRMAA (pronounced "Drama") or more formally the Distributed Resource Management Application API which allows us to hook into the Grid Engine from Java. The official Oracle overview should be included along with this documentation. The full javadoc for this API is located on StarExec under /cluster/sge-6.2u5/doc/javadoc.
TODO: This may change? We may be using a daemon or chron job to do this.
Starexec utilizes HTML 5 and CSS3. What this means is we use a simplified DOM and advanced selectors for styling pages. We also utilize open source scripts and style sheets to make StarExec compatible with non-html5 browsers. More information can be obtained here.
Here is an example of how to make an AJAX post request with jQuery. The method below can also be called with
If you are using a
$.get and the return type is JSON, you can shorthand this as follows:
Unfortunately, a similar method does not exist if you are using a
Google-gson is a Java library that allows us to serialize any Java object into a JSON string that can be sent back to the client to process. This way, we won't have to manually build a JSON string to send back ourselves. The gson library is released under the Apache License 2.0. For example if we use an AJAX call to get all the names in our database, and the Java servlet that is handling the request retrieves all the people in the database as "Person" objects then we will need to be able to convert this into a single string to send back to the client. gson will create this string for us which sends back the JSON representation of the list of people, and jQuery on the client side allows us to easily retrieve the information in the JSON string.
Sometimes you don't want to serialize an entire object to be sent to the client. For example we certainly don't want to be sending a password or confidential information to the client. To control which information is serialized into Json we have one of two options. Gson will ignore null fields by default, so we can null out a field to prevent it from being sent (obviously if a field doesn't have a value it can't be serialized!). Another option is to use gson's built in
@Expose annotation. We place this on a field to explicitly allow it to be serialized. All other fields are ignored. When in our serializer we have to explicitly say to ignore all fields that aren't exposed.
RESTEasy is a Java framework for handling restful web services. Restful web services (representational state transfer) are services that are invoked via http requests with existing HTTP methods such as POST, GET, PUT or DELETE. A service is requested via a standard service URL, and the data is returned in a known format as an HTTP response (in our case, JSON). All services are exposed in the StarExec service URL at
Requests come into to the
com.starexec.app.RESTServices servlet and are processed in methods decorated with special annotations that RESTEasy recognizes. Here is an example of a RESTEasy web service method:
@GET indicates the method accepts only HTTP GET requests.
@Path denotes the URL (relative to the service url seen above) to invoke the service, and
@Produces represents the type of the data being returned to the service caller which as we've stated will always be JSON for our purpose. The @QueryParam annotation represents a querystring parameter. This example method takes an id of a user to get information for through a query param. So to get the user with id 5, we would call the service like so:
RESTEasy also supports an alternative way of passing information besides query params. They can also be passed directly as part of the URL. This is the preferred method however query params may need to be used in special cases such as when using third party jQuery plugins on the client. Here is an example of a RESTEasy method that supports sending an ID as part of the path. Notice we now use
@PathParam instead of
@QueryParam and we use curly brackets in the path to denote the corresponding parameter.
Now if we wanted all job pairs of job number 4, we would call the service like so:
All backend data is stored in a MySQL database (the version changes as new updates are pushed out from Oracle). We make extensive use of stored procedures for efficiency and performance. We use JDBC to make calls to the database from Java.