segunda-feira, 2 de setembro de 2024

The Bluesky REST API

 You may have heard that Twitter has been blocked in Brazil. It has been defying local authorities and justice blocked its access from Brazil.

Have that said, most Brazilians migrated to Bluesky, the social network created by Twitter's founder. I moved to there as well (feel free to follow me, be aware I post a lot in Portuguese) and I am amazed with two things:

* The REST API is well documented and easy to use;

* Bluesky supports The AT Protocol, which seems to be great alternative of how we consume content from the internet (which is currently dominated by suspicious algorithms).


In this post let's reach the API to query about Java and build a dashboard out of it. 


The API Endpoint

We will be using the app.bsky.feed.searchPosts endpoint. It returns a JSON in the following format


{
"posts": [
{
"uri": "at://did:plc:xsqyottmupmtgs7q3quar6yx/app.bsky.feed.post/3l36s7n76bv2x",
"cid": "bafyreid43mz5v7lkck3om3wbwlcje2dddv2wsru32x5mqupfke5a7jdg6e",
"author": {
"did": "did:plc:xsqyottmupmtgs7q3quar6yx",
"handle": "wanningcatboy.bsky.social",
"displayName": "sam - ʷᵃⁿⁿⁱⁿᵍ ᵐⁱⁿʰᵃ ᵛⁱᵈᵃ",
"avatar": "https://cdn.bsky.app/img/avatar/plain/did:plc:xsqyottmupmtgs7q3quar6yx/bafkreif7ulgduagwph4jim5y34irfkcwgzfjwktejgvjnvnur4zw4hae7e@jpeg",
"associated": {
"chat": {
"allowIncoming": "following"
}
},
"labels": [],
"createdAt": "2024-08-29T22:00:51.821Z"
},
"record": {
"$type": "app.bsky.feed.post",
"createdAt": "2024-09-02T17:12:20.219Z",
"langs": [
"pt"
],
"reply": {
"parent": {
"cid": "bafyreibt6uol25amqmam4ekb6ybj2y3o4da7wtloe6osbwyv5rvcvemh3m",
"uri": "at://did:plc:jxgcpi3zvllx7fdd2hiigejy/app.bsky.feed.post/3l36gchfci72k"
},
"root": {
"cid": "bafyreibt6uol25amqmam4ekb6ybj2y3o4da7wtloe6osbwyv5rvcvemh3m",
"uri": "at://did:plc:jxgcpi3zvllx7fdd2hiigejy/app.bsky.feed.post/3l36gchfci72k"
}
},
"text": "Eu já entrei na facul com o objetivo de aprender logo java pq meu irmão eh dev pleno de Java e sempre fez propaganda pra mim kakakakakak"
},
"replyCount": 0,
"repostCount": 0,
"likeCount": 0,
"quoteCount": 0,
"indexedAt": "2024-09-02T17:12:20.219Z",
"labels": []
}
],
"cursor": "1"
}


To query it we need an API Key, which can be acquired going to your Profile Settings -> App Passwords -> Create App Password 



The API seems to enable CORS from any web app, hence for our application we will be using Dashbuilder

Our Application: A simple Dashboard

We used to use JavaFX or JavaFX Script on this blogs (e.g. Having fun creating a JavaFX App to Visualize Tweet Sentiments), but in this post we will be using a tool that I have been working until last year: Dashbuilder.

Dashbuilder allow us to write YAML that compiles to HTML pages and it is focused on Dashboard application, but is very flexible to render any external component, making it some sort of microfrontend tool.

Here are the steps we follow to create the dashboard:

* First we created the Dataset, which is basically the JSON above with a JSONAta expression to retrieve what we want. In our case we retrieved the author, author handle, text and the number of likes:

$.posts.[author.displayName, author.handle, record.text, likeCount]

* Then we extracted parameters from the dataset, such as the token, the search term and the API limit. This makes the YAML easily reusable;

* Finally we created the actual visualization. In this part you are free to create what you want from the data, since I was just testing it I simply created two barcharts to show who is talking more about the term and the most liked user and a table with the requests. Here' s the result:


You can see the result YAML on my github and you can even run it on Serverless Logic Webtools or in my personal Dashbuilder editor (which uses a more recent Dashbuilder version with Patternfly v5)


Thanks for reading!






sábado, 9 de outubro de 2021

Computer Vision with JavaFX and DJL

Computer Vision is not a new topic in Computer Science, but in the recent years it got a boost due the use of Neural Networks for classification, object detection, instance segmentation and pose prediction.

We can make use of these algorithms in Java using a Deep Learning library, such as DL4J as we already did in this blog with handwritten digits recognition, and detecting objects in a JavaFX application.

However, deep learning is popular mostly among Python developers (one reason is because Python easily wraps on top of native libraries, it will be easier in Java after Project Panama), so we have much more pre-trained models for Tensorflow and Pytorch libraries. It is possible to import them, but it requires a bit more of work then simply reusing a pre-trained model.

Fortunately there is a new library called Deep Java Library which offers a good set of pre-trained models. It makes use of Jupyter, which makes easier to try the library APIs. Another DJL feature is that it is made to wrap an existing library, so it works on top of Keras, Tensorflow, MXNet and other libraries.

In this post we will test some of the DJL Computer Vision models from a JavaFX application. Let's start first capturing webcam from JavaFX then use this input to a pre-trained model


Capturing Web Cam


The input data for the neural network we capture a webcam image. The project that worked without any issue with JavaFX on my Fedora 34 is capture-webcam. I used the JavaFX sample code and it just worked. See my workspace captured from the webcam:


Using Pre-trained Neural Networks

I started with a maven project that only used the webcam-capture. Then I added DJL maven dependencies and Eclipse allowed me to import the classes from DJL. Later I had also the ML library engine to run my models, this is how my final pom.xml looks like:


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.fxapps</groupId>
<artifactId>webcam-example-javafx</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>WebCam capture from a JavaFX application</name>
<properties>
<maven.compiler.source>16</maven.compiler.source>
<maven.compiler.target>16</maven.compiler.target>
<version.webcam.capture>0.3.12</version.webcam.capture>
<version.javafx>17-ea+16</version.javafx>
<djl.version>0.12.0</djl.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>ai.djl</groupId>
<artifactId>bom</artifactId>
<version>${djl.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- WebCam Capture -->
<dependency>
<groupId>com.github.sarxos</groupId>
<artifactId>webcam-capture</artifactId>
<version>${version.webcam.capture}</version>
</dependency>
<!-- OpenJFX -->
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>${version.javafx}</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-graphics</artifactId>
<version>${version.javafx}</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-media</artifactId>
<version>${version.javafx}</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-swing</artifactId>
<version>${version.javafx}</version>
</dependency>
<!-- ================ From DJL Examples ================ -->
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.4</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.12.1</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>ai.djl</groupId>
<artifactId>api</artifactId>
</dependency>
<dependency>
<groupId>ai.djl</groupId>
<artifactId>basicdataset</artifactId>
</dependency>
<dependency>
<groupId>ai.djl</groupId>
<artifactId>model-zoo</artifactId>
</dependency>
<!-- MXNet -->
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-model-zoo</artifactId>
</dependency>
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-engine</artifactId>
</dependency>
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-native-auto</artifactId>
<scope>runtime</scope>
</dependency>
<!-- Pytorch -->
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-engine</artifactId>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-native-auto</artifactId>
</dependency>
<!-- Tensorflow -->
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-model-zoo</artifactId>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-engine</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-cpu</artifactId>
<classifier>linux-x86_64</classifier>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-auto</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
</project>
view raw pom.xml hosted with ❤ by GitHub

Back to the code, what I wanted was to grab the image from the webcam, input into a pre-trained model, get the result and print in JavaFX instead printing the webcam image. To allow users to test the pre-trained models we added a combo box to the user interface. 


To abstract the model we created a abstract class called MLModel. This class wraps the model call, so we can focus on printing the image and the UI will not know about any specific model, making it easy to add and remove models. The class MLModel grabs the predictions from the ML algorithm and draw on the image accordingly to the result type:

package org.fxapps.predict;
import java.awt.image.BufferedImage;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.modality.cv.output.Joints;
import org.fxapps.predict.model.PoseEstimationPrediction;
public abstract class MLModel<T> {
public abstract String getName();
public T predict(BufferedImage image) {
var img = ImageFactory.getInstance().fromImage(image);
return predict(img);
}
protected abstract T predict(Image image);
public BufferedImage predictAndDraw(BufferedImage image) {
var img = ImageFactory.getInstance().fromImage(image);
T result = predict(img);
if (result instanceof Joints joints) {
img.drawJoints(joints);
}
if (result instanceof PoseEstimationPrediction poseEstimation) {
poseEstimation.personRect().ifPresent(rect -> {
img.getSubimage((int) rect.getX(),
(int) rect.getY(),
(int) rect.getWidth(),
(int) rect.getHeight())
.drawJoints(poseEstimation.joints());
});
}
if (result instanceof DetectedObjects objects) {
img.drawBoundingBoxes(objects);
}
// do not work on Android due hard code of BufferedImage
// change the use of BufferedImage so it should work on Android as well
return (BufferedImage) img.getWrappedImage();
}
@Override
public String toString() {
return getName();
}
}
view raw MLModel.java hosted with ❤ by GitHub

In the UI we have an array of implementations, which are selected when the combo box changes. We could use other ways to select the model, like Java Service Provider, but for our code we decided to keep it simple.


// objects to allow users to select a ML Model
ObjectProperty<MLModel<?>> selectedModel = new SimpleObjectProperty<>();
List<MLModel<?>> models = List.of(new ObjectDetectionMLModel(),
new InstanceSegmentationMLModel(),
new PoseEstimationMLModel());
AtomicBoolean runningPrediction = new AtomicBoolean();
// Creating the combo box and binding the selected model to what user selects
var modelOptions = new ComboBox<MLModel<?>>();
selectedModel.bind(modelOptions.getSelectionModel().selectedItemProperty());
modelOptions.getItems().addAll(models);
modelOptions.getSelectionModel().select(0);
// later when there's a new image from webcam, we lock the prediction while other is running
// the code below runs on a thread separated from JavaFX main thread
while (!stopCamera) {
try {
if ((img = webCam.getImage()) != null) {
if (!runningPrediction.get()) {
runningPrediction.set(true);
img = selectedModel.get().predictAndDraw(img);
runningPrediction.set(false);
}
img.flush();
ref.set(SwingFXUtils.toFXImage(img, ref.get()));
Platform.runLater(() -> imageProperty.set(ref.get()));
}
} catch (Exception e) {
e.printStackTrace();
}
}


Finally it was time to implement the algorithms itself. First we created MLModel implementation for Object detection. It returns a class of type DetectedObject and we re able to build the pre-trained model as we want. We could select the engine, select the data used to train the model and other parameters using the class ai.djl.repository.zoo.Criteria. In our case we selected from the Engine Tensorflow a model that was trained using the mobilenet_v2 dataset. A video is on my twitter.


package org.fxapps.predict;
import java.io.IOException;
import ai.djl.Application;
import ai.djl.MalformedModelException;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.training.util.ProgressBar;
import ai.djl.translate.TranslateException;
public class ObjectDetectionMLModel extends MLModel<DetectedObjects> {
private Predictor<Image, DetectedObjects> predictor;
public ObjectDetectionMLModel() {
try {
predictor = buildCriteria().loadModel().newPredictor();
} catch (ModelNotFoundException | MalformedModelException | IOException e) {
throw new RuntimeException("Not able to load detect object models", e);
}
}
protected Criteria<Image, DetectedObjects> buildCriteria() {
return Criteria.builder()
.optEngine("TensorFlow")
.optApplication(Application.CV.OBJECT_DETECTION)
.setTypes(Image.class, DetectedObjects.class)
.optFilter("backbone", "mobilenet_v2")
.optArgument("threshold", "0.2")
.optProgress(new ProgressBar())
.build();
}
@Override
public String getName() {
return "Object Detection";
}
@Override
public DetectedObjects predict(Image image) {
try {
return predictor.predict(image);
} catch (TranslateException e) {
throw new RuntimeException("Not able to detect objects", e);
}
}
}




For Pose Prediction we had to internally run two models: the first to extract Person from the input image and the other to do the PoseEstimation itself. We also had to calculate the points of the pose relative to the input image. See a video for this model

package org.fxapps.predict;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collections;
import java.util.Optional;
import ai.djl.Application;
import ai.djl.MalformedModelException;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.modality.cv.output.Joints;
import ai.djl.modality.cv.output.Rectangle;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.training.util.ProgressBar;
import ai.djl.translate.TranslateException;
import org.fxapps.predict.model.PoseEstimationPrediction;
public final class PoseEstimationMLModel extends MLModel<PoseEstimationPrediction> {
private Predictor<Image, DetectedObjects> objectsPredictor;
private Predictor<Image, Joints> posePredictor;
private static Joints EMPTY_JOINTS = new Joints(Collections.emptyList());
public PoseEstimationMLModel() {
super();
try {
// more precision
// objectsPredictor =
// Criteria.builder()
// .optApplication(Application.CV.OBJECT_DETECTION)
// .setTypes(Image.class, DetectedObjects.class)
// .optFilter("size", "512")
// .optFilter("backbone", "resnet50")
// .optFilter("flavor", "v1")
// .optFilter("dataset", "voc")
// .optProgress(new ProgressBar())
// .build()
// .loadModel()
// .newPredictor();
// faster
objectsPredictor = Criteria.builder()
.optEngine("TensorFlow")
.optApplication(Application.CV.OBJECT_DETECTION)
.setTypes(Image.class, DetectedObjects.class)
.optFilter("backbone", "mobilenet_v2")
.optArgument("threshold", "0.1")
.optProgress(new ProgressBar())
.build()
.loadModel()
.newPredictor();
} catch (ModelNotFoundException | MalformedModelException | IOException e) {
throw new RuntimeException("Not able to load object detector model.");
}
try {
posePredictor = Criteria.builder()
.optApplication(Application.CV.POSE_ESTIMATION)
.setTypes(Image.class, Joints.class)
.optFilter("backbone", "resnet18")
.optFilter("flavor", "v1b")
.optFilter("dataset", "imagenet")
.build().loadModel().newPredictor();
} catch (ModelNotFoundException | MalformedModelException | IOException e) {
throw new RuntimeException("Not able to load pose estimatino model.");
}
}
@Override
public String getName() {
return "Pose Estimation";
}
@Override
protected PoseEstimationPrediction predict(Image image) {
try {
var personPosOp = retrievePerson(image);
var personJoints = personPosOp.map(rect -> predictPose(image, rect))
.orElse(EMPTY_JOINTS);
return new PoseEstimationPrediction(personPosOp, personJoints);
} catch (TranslateException e) {
e.printStackTrace();
return new PoseEstimationPrediction(Optional.empty(), EMPTY_JOINTS);
}
}
private Optional<Rectangle> retrievePerson(Image img) throws TranslateException {
var detectedBoxes = objectsPredictor.predict(img);
// use to draw the predicted objects
// img.drawBoundingBoxes(detectedBoxes);
return detectedBoxes.items()
.stream()
.map(i -> (DetectedObjects.DetectedObject) i)
.filter(item -> "person".equalsIgnoreCase(item.getClassName()))
.findFirst()
.map(box -> extractPersonBounds(img, box));
}
private Rectangle extractPersonBounds(Image img, DetectedObjects.DetectedObject box) {
var rect = box.getBoundingBox().getBounds();
int width = img.getWidth();
int height = img.getHeight();
int personX = (int) (rect.getX() * width);
int personY = (int) (rect.getY() * height);
int personWidth = (int) (rect.getWidth() * width);
int personHeight = (int) (rect.getHeight() * height);
if (personX > personWidth) {
personX = personWidth;
}
if (personY > personHeight) {
personY = personHeight;
}
if (personX < 0) {
personX = 0;
}
if (personY < 0) {
personY = 0;
}
int rectXBound = personX + personWidth;
int rectYBound = personY + personHeight;
if (rectXBound > img.getWidth()) {
personWidth = personWidth - (rectXBound - img.getWidth());
}
if (rectYBound > img.getHeight()) {
personHeight = personHeight - (rectYBound - img.getHeight());
}
return new Rectangle(personX, personY, personWidth, personHeight);
}
private Joints predictPose(Image image, Rectangle rect) {
try {
var personImage = image.getSubimage((int) rect.getX(),
(int) rect.getY(),
(int) rect.getWidth(),
(int) rect.getHeight());
return posePredictor.predict(personImage);
} catch (TranslateException e) {
e.printStackTrace();
return EMPTY_JOINTS;
}
}
// use this for debug purpose
protected static void saveJointsImage(Image img, Joints joints) {
Path outputDir = Paths.get("build/output");
try {
Files.createDirectories(outputDir);
img.drawJoints(joints);
Path imagePath = outputDir.resolve("joints.png");
// Must use png format because you can't save as jpg with an alpha channel
img.save(Files.newOutputStream(imagePath), "png");
} catch (IOException e) {
e.printStackTrace();
}
}
}




Finally we also have Instance Segmentation, but it was so, so slow that I will not discuss it here. You are free to run the application and test it.



Conclusion

Java is a good alternative for building Machine Learning applications! DJL and its great model zoo makes it easier to reuse ms trained with other libraries. Next steps would be use other family of trained models, such as NLP and create more useful applications, like augmented reality!

Full code on Github.


quinta-feira, 10 de junho de 2021

Database Query Server using Quarkus



We are updating our application to Quarkus and I have this requirement of supporting datasouces that users of our application can create.

This is already supported in Widlfly, but does not seem to be supported on Quarkus. Before implementing this in our system we created a PoC which we share in this post.


The problem

Agroal is the part of Quarkus responsible for DataSources and pool connections. It is easy to configure datasources in Quarkus, but some properties only works during development time, after the JAR for distribution is built then you can't change some properties.

The solution is programatically create datasources so the application is flexible in a way that we can add datasets using system properties. We also must make sure that all supported data base drivers are in the application so users just needs to setup properties and no other action is required.

All drivers dependencies

Datasource from property

We can create datasources using a map of properties. The properties keys are defined in class AgroalPropertiesReader. When creating a properties reader we can specify a prefix which will be used to read the properties from the source.


To build the prefix we need all datasources mapped by the user followed by the properties setup for each mapped datasource. We can simply support the same properties as AgroalPropertiesReader. Here's a sample configuration


From Java we use Microprofile config to collect the datasource properties to a map, set the correct prefix and then build the properties that will be used to create a datasource. This is done right after the application is started so any error in configuration prevents the service to run.


import java.sql.SQLException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.stream.StreamSupport;
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;
import javax.inject.Inject;
import javax.sql.DataSource;
import io.agroal.api.AgroalDataSource;
import io.agroal.api.configuration.supplier.AgroalPropertiesReader;
import io.quarkus.runtime.StartupEvent;
import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.inject.ConfigProperty;
@ApplicationScoped
public class DataSourceLoader {
private static final String DATASOURCES = "datasources";
private static final String DATASOURCE = "datasource";
private static final String PREFIX_TEMPLATE = DATASOURCE + ".%s.";
@Inject
Config config;
@ConfigProperty(name = DATASOURCES, defaultValue = "")
Optional<List<String>> datasourcesProp;
Map<String, DataSource> registeredDataSources;
void load(@Observes StartupEvent startup) throws SQLException {
var allProps = new HashMap<String, String>();
var datasources = datasourcesProp.orElse(List.of());
registeredDataSources = new HashMap<>();
StreamSupport.stream(config.getPropertyNames().spliterator(), false)
.filter(p -> p.startsWith(DATASOURCE))
.forEach(k -> allProps.put(k, config.getValue(k, String.class)));
for (String ds : datasources) {
var prefix = PREFIX_TEMPLATE.formatted(ds, AgroalPropertiesReader.JDBC_URL);
var agroalProps = new AgroalPropertiesReader(prefix);
agroalProps.readProperties(allProps);
registeredDataSources.put(ds, AgroalDataSource.from(agroalProps.get()));
}
}
public Optional<DataSource> getDataSource(String name) {
return Optional.ofNullable(registeredDataSources.get(name));
}
public Set<String> datasources() {
return registeredDataSources.keySet();
}
}



Running Queries


Now that we map all datasources we simply have to expose it via REST to run queries using pure JDBC code. The trick is to transform any query result to a suitable data structure that can be translated to JSON, for this we use a MAP of LIST (the best data structure ever), where the key is the column name and the list are the rows for that specific column:


import java.sql.Connection;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.Status;
import org.fxapps.datasource.DataSourceLoader;
@Path("datasource")
@Produces(MediaType.APPLICATION_JSON)
public class DataSourceResource {
@Inject
DataSourceLoader loader;
@GET
public Collection<String> list() {
return loader.datasources();
}
@POST
@Path("{dsName}/query")
public Response executeQuery(@PathParam("dsName") String dsName, String query) throws SQLException {
var dataSourceOp = loader.getDataSource(dsName);
if (dataSourceOp.isEmpty()) {
return Response.status(Status.NOT_FOUND).build();
}
var ds = dataSourceOp.get();
var connection = ds.getConnection();
var result = extractResult(query, connection);
connection.close();
return Response.ok(result).build();
}
private HashMap<String, List<String>> extractResult(String query, Connection connection) throws SQLException {
var result = new HashMap<String, List<String>>();
try (var stmt = connection.createStatement()) {
var rs = stmt.executeQuery(query);
var meta = rs.getMetaData();
var nColumns = meta.getColumnCount();
while (rs.next()) {
for (int i = 1; i <= nColumns; i++) {
var column = meta.getColumnName(i);
var value = rs.getString(i);
result.putIfAbsent(column, new ArrayList<String>());
result.get(column).add(value);
}
}
}
return result;
}
}


Now we can start our application and send requests to execute queries using HTTP requests:


curl --silent --data 'select id, variableId, value from VariableInstanceLog' http://localhost:8080/datasource/ds1/query



Finally you can run the built JAR using system properties and it will connect to any database:

java -Ddatasources=ds1,ds2 \

-Ddatasource.ds1.jdbcUrl=jdbc:mariadb://localhost:3306/jbpmdb \

-Ddatasource.ds1.providerClassName=org.mariadb.jdbc.Driver \

-Ddatasource.ds1.maxSize=10 \

-Ddatasource.ds1.principal=jbpm \

-Ddatasource.ds1.credential=jbpm \

-Ddatasource.ds2.jdbcUrl=jdbc:mariadb://localhost:3306/test \

-Ddatasource.ds2.providerClassName=org.mariadb.jdbc.Driver \

-Ddatasource.ds2.maxSize=10 \

-Ddatasource.ds2.principal=repasse \

-Ddatasource.ds2.credential=repasse \

-jar target/configurable-datasource-1.0.0-SNAPSHOT-runner.jar


Conclusion

In this post we show how we can use Agroal and quarkus to create an application that can run queries on any of the supported databases. The code is in my github

This application has room for a lot of improvements:


  • Create a UI
  • Better return error messages from REST endpoint
  • Add tests
  • Extend for other commands (INSERT, UPDATE and so on)
  • Support new data sources creation after the application is running (This could be useful for creating a generic Database Client)


Feel free to send PRs if you implement any of these improvements!




sábado, 14 de novembro de 2020

Creating Fat JARs for JavaFX Applications

A FAT JAR is a type of Java application distribution where a single JAR contains all other dependencies, so no additional file is required to run the JAR besides the Java Virtual Machine.

For any Java maven based application, creating a FAR JAR could be solved by using Maven Shade Plugin. However, creating FAT Jars using JavaFX may be a challenge because JavaFX uses modules.

Fortunately this subject was intensely discussed in the WEB, and a good explanation and a solution was provided by Jose Pereda in this StackOverflow response.

In this post I want to briefly share the steps to make a FAT JAR and post an example on my github so I can point others to check the example.


How to create a FAT JAR for a JavaFX application?

1- Create a main class that will run your application. This class must have the main method and call your actual application static launch method;

2- Add the Shade Plugin to your project. For those using Gradle notice that Jose Pereda also provided an answer about it in Stack Overflow;

3- In the shade plugin configuration make sure you are setting the Main class to be the one created in step 1.


That's basically all you need. If i is not clear you can check my sample project in github. The three files, App.java, Main.java and pom.xml can be checked below.


package org.fxapps.javafx.fatjar;
import javafx.animation.Animation;
import javafx.animation.FadeTransition;
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.effect.InnerShadow;
import javafx.scene.layout.StackPane;
import javafx.scene.paint.Color;
import javafx.scene.text.Font;
import javafx.scene.text.FontWeight;
import javafx.stage.Stage;
import javafx.stage.StageStyle;
import javafx.util.Duration;
public class App extends Application {
public static void main(String[] args) {
launch();
}
@Override
public void start(Stage stage) throws Exception {
var lblHello = new Label("Hello World!");
var fd = new FadeTransition(Duration.millis(900), lblHello);
// Label Setup
lblHello.setFont(Font.font(Font.getDefault().getFamily(), FontWeight.BOLD, 60));
lblHello.setEffect(new InnerShadow());
lblHello.setTextFill(Color.GOLD);
lblHello.setOnMouseClicked(e -> System.exit(0));
// Animation Setup
fd.setAutoReverse(true);
fd.setFromValue(1.0);
fd.setToValue(0.2);
fd.setCycleCount(Animation.INDEFINITE);
fd.play();
stage.setScene(new Scene(new StackPane(lblHello)));
stage.initStyle(StageStyle.UNDECORATED);
stage.show();
}
}
view raw App.java hosted with ❤ by GitHub
package org.fxapps.javafx.fatjar;
// This is the Main used by the shade plugin.
public class Main {
public static void main(String[] args) {
App.main(args);
}
}
view raw Main.java hosted with ❤ by GitHub
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.fxapps.javafx.fatjar</groupId>
<artifactId>fatjar-javafx</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Fat Jar JavaFX</name>
<description>Project to build a JavaFX 11+ Fat JAR.</description>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.release>11</maven.compiler.release>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<javafx.version>11.0.2</javafx.version>
<javafx.plugin.version>0.0.4</javafx.plugin.version>
<main.class>org.fxapps.javafx.fatjar.Main</main.class>
</properties>
<dependencies>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-base</artifactId>
<version>${javafx.version}</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-graphics</artifactId>
<version>${javafx.version}</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>${javafx.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.openjfx</groupId>
<artifactId>javafx-maven-plugin</artifactId>
<version>${javafx.plugin.version}</version>
<configuration>
<mainClass>${main.class}</mainClass>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${main.class}</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
view raw pom.xml hosted with ❤ by GitHub