diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..83ab983f --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,14 @@ +# Architecture decision record (ADR) + +This folder contains some topics that they motivated some decisions +in this project. + +* [Why this project was started](adr-lejos-support.md) +* [Logging support](adr-logging-support.md) +* [New Display API](adr-display-api.md) +* [Changes to leJOS sensor API](adr-lejos-sensor.md) +* [Support for newer Java versions](adr-openjdk-builds.md) + +## References: + +https://www.thoughtworks.com/radar/techniques/lightweight-architecture-decision-records diff --git a/docs/adr/adr-display-api.md b/docs/adr/adr-display-api.md new file mode 100644 index 00000000..8b042bb1 --- /dev/null +++ b/docs/adr/adr-display-api.md @@ -0,0 +1,107 @@ +# ev3dev Display API (October 2018) + +## Reason + +In Linux 4.14 for EV3, there was a change in the LCD framebuffer code +which required a redesign of our display architecture. The change was +that kernel no longer accepted 1bpp black-and-white bitmap. Now a +32bpp XRGB bitmap was required. To allow for simultaneous support of +these two formats and also to address a few other issues, a new +display API was designed. + +## API Design + +The code has roughly these interfaces: +* leJOS LCD emulation in `LCD` - This reimplements some parts of leJOS + `GraphicsLCD` API on top of Java2D Graphics2D. The `BufferedImage` is + obtained from and the refresh is forwarded to `JavaFramebuffer`. + The `JavaFramebuffer` itself is obtained from `SystemDisplay`. +* Glue in `SystemDisplay` - This provides a factory for a system + display manager - either a native one or a remote one, and + a JavaFramebuffer above the display manager. +* `JavaFramebuffer` interface - This interface abstracts away + a framebuffer. The interface provides: + * basic information about the display geometry, + * methods for creating offscreen `BufferedImage` for drawing, + * writing full-size offscreen buffers to the display, + * clearing the display, + * saving and restoring a single snapshot, + * getter for the underlying `DisplayInterface` +* `DisplayInterface` interface - This class manages the underlying + display output. It provides: + * switching between text mode (Linux console is displayed) and + graphics mode (user graphics is displayed), + * framebuffer creation - this class creates the appropriate + `JavaFramebuffer` implementation for the system display. + * framebuffer lifetime management - this class stores the opened framebuffer. +* `FramebufferProvider` interface with `AllImplFailedException` - + These classes provide Java SPI for framebuffer formats. + The `DisplayInterface` uses this SPI to instantiate the correct + `JavaFramebuffer` implementation for the system framebuffer. + +Then, we have the following implementations: +* `ImageUtils` class - This class implements BufferedImage creation + for 32bpp XRGB and 1bpp BW framebuffers. +* `LinuxFramebuffer` class - This class implements `JavaFramebuffer` + for native Linux framebuffer device. +* `BitFramebuffer`, `RGBFramebuffer` - These classes specialize + `LinuxFramebuffer` for 1bpp BW and 32bpp XRGB formats. +* `BitFramebufferProvider`, `RGBFramebufferProvider` - These classes wrap + `BitFramebuffer` and `RGBFramebuffer` for usage in `FramebufferProvider`. +* `OwnedDisplay` class - This class implements `DisplayInterface` + for standard Linux console. The Java process must have access to + the system console. +* `StolenDisplay` class - This class implements `DisplayInterface` when + the standard Linux console is unavailable. It ignores text/graphics mode + switching. However it disables brickman process when instantiated. + +Under all of this is an extension of leJOS Native* APIs: +* `NativeFile` class - this class wraps ILibc file system calls. +* `NativeDevice` class - this class wraps ILibc system calls + for Linux devices. +* `NativeFramebuffer` class - this class wraps ILibc system calls + for Linux framebuffer devices. +* `NativeTTY` class - this class wraps ILibc system calls + for Linux console devices. +* `NativeConstants` class - contains basic C macros, like errno values and + ioctl arguments. +* `ILibc` interface - abstraction of basic system calls. + This is useful for testing the interface code. +* `NativeLibc` interface - class which is used for JNA binding to system libc. + +`ILibc` interface had been mocked for purposes of unit testing: +* `EmulatedLibc` - class which provides a virtual file system to which + fake files and devices can be mounted. +* `IFile` - interface for fake files, +* `ICounter` - interface for event counters, +* `CountingFile` - this class wraps another IFile and provides + counting of system call invocations, +* `EmulatedFramebuffer` - this class provides a mocked version of system + framebuffer device. +* `EmulatedFramebufferBuilder` - this class helps with `EmulatedFramebuffer` + construction. + +This design makes it possible to implement new framebuffer formats +(through subclassing JavaFramebuffer and FramebufferProvider) and +new output devices (through subclassing DisplayInterface and modifying +SystemDisplay). It also allows for unit testing the interfacing code +through abstracted system call interface. + +For example, it should be possible to relatively easily implement a Swing-backed +display output. This feature might actually be implemented in a future +version of the display code. + +## Running programs + +There is also a change in how the programs have to be launched. Until now, +simply running the JAR on the console was sufficient. However this wasn't +compliant with how ev3dev handles console (the old method is still possible +through current StolenDisplay). + +For this reason, we introduced the recommended usage of `brickrun` +to our project too. This program allocates a virtual terminal +for the application and suspends Brickman. Then, it launches there +the application it was given in its arguments. + +When you run the application from Brickman menu, using brickrun is not +necessary, because Brickman automatically suspends itself. diff --git a/docs/adr/adr-lejos-sensor.md b/docs/adr/adr-lejos-sensor.md new file mode 100644 index 00000000..7a36d70b --- /dev/null +++ b/docs/adr/adr-lejos-sensor.md @@ -0,0 +1,29 @@ +# leJOS Sensor API changes (Nov 2015) + +In this project, we reimplemented most of the leJOS sensor API. +However, there are some incompatibilities and changes to enhance +performance and decrease the complexity of the sensor code +that break previously established promises: + +* Using multiple `SensorMode`s in parallel doesn't work anymore. + This is because for now, we have removed the kernel mode switching + from `fetchSample()` of sensor modes. Instead, the mode switch + is performed in the associated `get*Mode()` functions. This means + that once you call one `get*Mode()` function, you cannot use + `SensorMode`s from the other ones. This also means that once you + reset the `EV3GyroSensor`, you have to call `get*Mode()` once more. + + In the future, if we find a way how to do this efficiently and + without making it omplex, we might add support for this + feature again. However, we might also keep it the way it is now, + as this establishes a different promise - `fetchSample` only fetches + data, and it doesn't touch the sensor mode. + +* Most of the `BaseSensor` functions dealing with sensor modes + are effectively broken now. This is caused by the previous point - + sensor mode is not switched when using these functions. + +* Waiting for multiple values does not work. We do not know how to + wait for a new value in ev3dev. This means that IR sensor function + reading multiple commands will read only one, otherwise it will + throw an exception. diff --git a/docs/adr/adr-lejos-support.md b/docs/adr/adr-lejos-support.md new file mode 100644 index 00000000..1efc28a4 --- /dev/null +++ b/docs/adr/adr-lejos-support.md @@ -0,0 +1,26 @@ +# Implement LeJOS API on top of EV3Dev (Nov 2015) + +EV3 Brick is the first Lego Mindstorms Brick to run a Linux-based operating system. + +LeJOS team designed a solution on top of a LEGO-provided Linux userspace & +Linux kernel with modules also provided by LEGO, although enhanced by leJOS hackers. +This architecture doesn't have proper USB support, so devices like USB LIDAR or an Arduino Board do not work. +Also, LEGO didn't provide any package manager, so the core system is effectively stuck in time. + +As the time passed, the number of developers decreased and the project development slowed down. + +In 2015 a project named EV3Dev was launched, am OSS project designed to provide a +Debian distribution for EV3 and RaspberryPi boards like BrickPi or PiStorms. +The project doesn't use the userspace nor the kernel modules from LEGO. +Rather, the sensor and actuator interfaces were reimplemented to provide +a better and easier-to-understand interface. This includes EV3 sensors +and also various other sensors. + +The main reasons to implement LeJOS Interfaces on top of EV3Dev were: + +- Lack of LeJOS releases. Last release was launched in November of 2015 +- Lack of a complete Linux distro support, +- Lack of USB support +- Lack of support for modern JVM + +That reasons motivated the creation of this project. diff --git a/docs/adr/adr-logging-support.md b/docs/adr/adr-logging-support.md new file mode 100644 index 00000000..fde5204b --- /dev/null +++ b/docs/adr/adr-logging-support.md @@ -0,0 +1,32 @@ +# Logging Support (Nov 2015) + +We recommend to move away from using `System.out.println("Your message")` +because using `stdout` is not the same as using `stderr` or a logging interface. + +The reason you shouldn't use `System.out` is that we depend on output redirection +provided by ev3dev's brickrun. `stderr` is redirected via SSH to the user +when running remotely. If you use `System.out`, the message will be +displayed on the brick display only. On the other hand, this might +be intentional, in which case you should use `System.out`. + +Also, to align with the best practises in the software industry, we +integrated support for SLF4J (Simple Logging Facade for Java). + +The Simple Logging Facade for Java (SLF4J) serves as +a simple facade or abstraction for various logging frameworks +(e.g. java.util.logging, logback, log4j) allowing +the end user to plug in the desired logging framework at deployment time. + +The library uses SLF4J in the whole project, but the user has to +choose the final implementation. In the example, the development uses +Logback but users can choose any other compatible logging framework. + +For testing purposes, enabling traces is a good practice, but for +production builds you might want to disable some levels. +Generating too many tracing output creates a measurable impact on the program performance. + +## Links + +https://javarevisited.blogspot.com/2016/06/why-use-log4j-logging-vs.html +https://www.slf4j.org/ +https://logback.qos.ch/ diff --git a/docs/adr/adr-modular.md b/docs/adr/adr-modular.md new file mode 100644 index 00000000..be96442a --- /dev/null +++ b/docs/adr/adr-modular.md @@ -0,0 +1,31 @@ +# Modular design (Nov 2015) + +The project has been designed like a set of libraries stackables +to offer different solutions for different use cases with the Brick. + +## Libraries + +**Core** + +The main library offers a Integration solution with EV3Dev. +Using this library, it is possible to interact with +Sensors & Actuators from Lego Mindstorms & third party manufacturers +like MindSensors or Dexter Industries. + +This library depends of `lejos-commons` & `jna` + +**LeJOS // Commons** + +`lejos-commons` manage a set of Interfaces in order to maintain the +same way to develop than LeJOS but in this case using the `EV3Dev` runtime. + + +**LeJOS // Navigation** + +`lejos-navigation` is a library to use the local navigation stack from LeJOS. + +**USB-devices** + +This library has been designed to manage different usb devices like +Arduino boards, LIDARs, IMUs, Microphones, GPS + diff --git a/docs/adr/adr-openjdk-builds.md b/docs/adr/adr-openjdk-builds.md new file mode 100644 index 00000000..fdcb1ad3 --- /dev/null +++ b/docs/adr/adr-openjdk-builds.md @@ -0,0 +1,47 @@ +# Support for newer Java versions (April 2018) + +In December 2017, we noticed that Oracle had released their HotSpot port +for ARM32. Until that moment, there were only a few options of JVMs +which would run on the EV3: +* Debian-provided Zero build, which is a lot slower than HotSpot JIT. +* Oracle-provided EJDK7 build, which was used in leJOS. +* Oracle-provided EJDK8 build, which was used in leJOS as well. +* Some alternative JVMs, but we haven't finished investigating these options. + +The problem with EJDK8 from Oracle was that it was the last supported +Java release for EV3. At that time, Java 9 was already available and +Java 10 was coming as well. + +So, now that we knew OpenJDK 9 had the necessary bits from EJDK8, we +tried to create a build of OpenJDK for EV3. This resulted in creation +of [ev3dev-lang-java/openjdk-ev3][1] repository. +Over time, we adopted Docker for the builds. The next big thing was +that after some communication, [AdoptOpenJDK][2] Group +allowed us to use their [CI infrastructure][2] to do our builds. From that time, +instead of doing the releases on developer's computers, they are performed +on their Linux Jenkins slaves. + +To make OpenJDK work on EV3, we maintain a set of patches on top of +OpenJDK tree. They do these things: +* ARMv5 support - there are some memory barriers that work only on ARMv6+. They are disabled on ARMv5 now. +* Additional optimizations - GCC flags `-mcpu` is used to compile for the CPU in EV3. +* SoftFloat support - previously, a SoftFloat-2 library was injected into `arm-sflt` builds + to provide floating point emulation with higher precision for some code paths. This patch + adds support for a modified version of SoftFloat-3e to be used. +* Debian library path - this patch adds support for Debian-style `/usr/lib/jni` path and others. + +Also, with the new features in Java 9 and onwards, we changed Java packaging: +* JRI - Java Runtime Image - provides very small runtime environment suitable for ev3dev. +* JDK - Java Development Kit - provides full Java JDK build for EV3. +* JMODs - Java MODules - with these a curious user can build larger + subsets of full Java than our JRI is. + +There is a small problem with using Java HotSpot, though. ARMv5/softfp +is no longer a supported configuration. We depend on that Java developers +do not break or remove the support for this platform in future releases. +If such a thing happens, we will have to either use the Zero JVM, or to +use an alternative JVM altogether. + +[1]: https://github.com/ev3dev-lang-java/openjdk-ev3 +[2]: https://adoptopenjdk.net/ +[3]: https://ci.adoptopenjdk.net/ diff --git a/gradle/deploy.gradle b/gradle/deploy.gradle index 63ade9f5..cb96f2df 100644 --- a/gradle/deploy.gradle +++ b/gradle/deploy.gradle @@ -13,6 +13,50 @@ task testConnection { } } +task free { + doLast { + ssh.run { + session(remotes.ev3dev) { + println "free " + execute "free " + } + } + } +} + +task top { + doLast { + ssh.run { + session(remotes.ev3dev) { + println "top " + execute "top " + } + } + } +} + +task ps { + doLast { + ssh.run { + session(remotes.ev3dev) { + println "ps " + execute "ps aux | sort -n -k 4 " + } + } + } +} + +task stopBluetooth { + doLast { + ssh.run { + session(remotes.ev3dev) { + println "echo -e \"maker\" | sudo systemctl stop bluetooth" + execute "echo -e \"maker\" | sudo systemctl stop bluetooth" + } + } + } +} + task ev3devInfo { doLast { ssh.run { @@ -94,8 +138,8 @@ task remoteSudoRun { doLast { ssh.run { session(remotes.ev3dev) { - println "echo -e \"maker\" | sudo -S brickrun -- java -jar /home/robot/" + "${rootProject.name}" + "-" + version + "-all" + ".jar" - execute "echo -e \"maker\" | sudo -S brickrun -- java -jar /home/robot/" + "${rootProject.name}" + "-" + version + "-all" + ".jar" + println "echo -e \"maker\" | sudo -S java -jar /home/robot/" + "${rootProject.name}" + "-" + version + "-all" + ".jar" + execute "echo -e \"maker\" | sudo -S java -jar /home/robot/" + "${rootProject.name}" + "-" + version + "-all" + ".jar" } } } @@ -159,6 +203,10 @@ apply from: './gradle/config.gradle' //Organize tasks in a Group def groupName = "ev3dev-lang-java" testConnection.group = groupName +free.group = groupName +ps.group = groupName +top.group = groupName +stopBluetooth.group = groupName ev3devInfo.group = groupName removePreviousJar.group = groupName deploy.group = groupName