Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 05 2018

Ignite Realtime Blog: Smack 4.3.0 released

@flow wrote:

The Ignite Realtime developer community is happy to announce the availability of Smack 4.3.0.

This is the first release of the new Smack 4.3 series, which brings new features and bug fixes.

Compared to the API changes between Smack 4.1 and 4.2, the changes of the Smack 4.3 are far less intrusive. Remember that the 4.2 API changes where mostly caused by the introduction of the Jid type, replacing String which was previously used to represent a JID.

We like to thank everyone who contributed by reporting bugs or giving suggestions. A special thank goes to people who contributed code:

$ git shortlog -sn 4.2.4..4.3.0
   227  Florian Schmaus
    22  vanitasvitae
    14  Jesus Fuentes
     6  Paul Schaub
     3  Guus der Kinderen
     2  Timothy Pitt
     2  Yash Thakkar
     1  Boris Grozev
     1  Cem Yabansu
     1  Daniel Hintze
     1  Fernando Ramirez
     1  Mohsen Hariri
     1  Rajat Kumar Gupta
     1  Son Goku
     1  Thomas
     1  akrherz

More information about the 4.3.0 release can be found one the release’s Readme.

Posts: 1

Participants: 1

Read full topic

August 01 2018

Prosodical Thoughts: PEP improvements in trunk

Our next trunk nightly release uses a new implementation of PEP (XEP-0163) by default. PEP is used for a number of things, primarily sharing with your contacts: Extended statuses (e.g. sharing the user’s current playing music, their mood, activity, etc.) Avatars OMEMO keys Chatroom bookmarks While PEP started out as a simplified form of ‘pubsub’ and a way to share information with your contacts, it quickly became clear that it was also a neat mechanism for sharing public info with non-contacts (such as OMEMO keys) or sharing private data (such as saved chatrooms) with other clients on your account.

July 30 2018

Paul Schaub: Summer of Code: Smack has OpenPGP Support!

I am very proud to announce, that Smack got support for OpenPGP for XMPP!

Today the Pull Request I worked on during my GSoC project was merged into Smacks master branch. Admittedly it will take a few months until smack-openpgp will be included in a Smack release, but that gives me time to further finalize the code and iron out any bugs that may be in there. If you want to try smack-openpgp for yourself, let me know of any issues you encounter :)

(Edit: There are snapshot releases of Smack available for testing)

Now Smack does support two end-to-end encryption methods, which complement each other perfectly. OMEMO is best for people that want to be safe from future attacks, while OpenPGP is more suited for users who appreciate being able to access their chat history at any given time. OpenPGP is therefore the better choice for web based applications, although it is perfectly possible to implement web based clients that do OMEMO (see for example the Wire web app, which does ratcheting similar to OMEMO).

What’s left to do now is updating smack-openpgp due to updates made to XEP-0373 and extensive testing against other implementations.

Happy Hacking!

July 27 2018

Paul Schaub: Summer of Code: Finalizing the PR

Quick update:

Only a few days are left until the last and final Evaluation Phase.

I spent the week opening my pull request against Smacks master branch and adding a basic trust management implementation. Now the user is required to make decisions whether to trust a contacts key or not. However, the storage implementation is kept very modular, so an implementor can easily create a trust store implementation that realizes custom behaviour.

Smack-openpgp now allows users which did not subscribe to one another to exchange encryption keys quite easily. If a user receives an encrypted message, the implementation automatically fetches the senders keys to allow signature verification.

Furthermore there are more JUnit tests now, so that Smacks total test coverage actually increases when my PR gets merged :D

Happy Hacking!

July 25 2018

Erlang Solutions: MongooseIM 3.1 - Inbox got better, testing got easier

This summer the MongooseIM team have not had a second to be lazy - the development of MongooseIM 3.1 release took over what was supposed to be our downtime, when all major activities driven throughout the year slow down. We’re happy to say that in this time of leisure, the new release is packed with important features, improvements and excitement. Take a look at the version highlights and be part of this major step in creating a world-class communication server.

This time, a large part of the changes are focused on development efficiency, but rest assured, we’ve added new items you are going to appreciate.

In the “Test Runner” section, you get to learn all about a little tool we provided so that you can easily set the development environment up and run tests locally without Travis CI.

Our Inbox extension has got three big updates that push the boundaries of what’s currently offered in XMPP world. As a rule, we want to lead by example; less talk and more action. That is why working in cooperation with Forward, we decided to put forward an implementation of a popular use case as a unique feature in this software sector.

This release is also an important lesson for us. A lesson about edge, cases and concurrency in integration testing. You don’t necessarily have to be an Erlang developer to benefit from the third section, but reading it allows you to learn with us.

The “Honorable Mentions” section may seem minor, but for some projects the items listed there can indeed make a difference! It’s a candy mix of different changes, so read carefully not to miss your favourite flavours!

Obviously, a single blog post is too small a space to tell a profound story about every new item in the changelog, so we encourage you to check it out. You can find a link at the bottom of this blog post.

Test Runner

The Travis CI is our main verification tool for the Pull Requests we create and merge. Whilst being convenient and critical for our process, it is not very handy for day-to-day development. It is very common to frequently execute limited subset of tests to ensure that a new piece of code we wrote works perfectly. However, waiting for Travis results every time would extend implementation time excessively as it executes multiple presets, while being a shared resource at the same time.

The test runner is a script that helps to set the development environment up, and run tests on a developer machine locally. The test runner shares a lot of code with our Travis CI build scripts, which ensures that test results are consistent between local and CI runs.

The test runner allows to choose which tests to run. It can be useful, when one of the tests is failing and we want to rerun it.

Since MongooseIM supports different database backends, the test runner is able to set a selected database up for you and configure MongooseIM. Simply put, it prepares the local environment before executing actual test suites.

The test runner supports a lot of options to customise your build and test execution scenario. It also supports shell autocompletion for option names and values.

We’ve prepared a recording for you that presents a basic usage of the script.

.

Please note that Asciinema allows you to pause and copy any text fragment you like, so it would be very easy for you to repeat the same steps.

New Inbox Features

Originally sponsored by and created for Forward, Inbox has been available as an open source extension for the last two months already. In MongooseIM 3.1, it returns with a bundle of fresh goodies. The first feature is MSSQL support.

Despite being less frequently used with MongooseIM compared to MySQL or PostgreSQL, it’s still an important piece of the architecture for many projects, especially those running in Azure cloud. We don’t want you to feel excluded, dear Microsoft users!

The second one is the support for classic MUC group chats. MUC Light was never intended as a complete replacement for original XEP-0045 solution. It means that numerous projects exists where mod_muc is still a better match than its younger sibling, and they may now benefit from inbox features as well!

Last but not least is the timestamp support. First of all, they are stored in DB and returned in Inbox query results. For those using mod_inbox from MIM 3.0: you’ll need to update your schemas but don’t worry - it isn’t very complicated. What’s more, a client may now request conversations from a certain time period and sort them by timestamp, both ascending and descending.

This is not our final word on this matter. You may expect further improvements to this extension in upcoming MongooseIM versions!

We’ve prepared a DEMO of the Inbox feature. It shows both the backed and the frontend side of it. The application uded in our DEMO is designed by Forward.

Lessons learnt regarding CI

OK, these are short and sweet but nevertheless important:

  1. Avoid RPC in integration tests. They tend to time out in slow CI environments (such as Travis).
  2. When test users exchange messages, always wait until they are received to ensure proper synchronisation.
  3. On a slow machine, MSSQL SELECT query may return more than one row (even when retrieving by the exact primary key value) as a consequence of the transaction deadlock.
  4. When you can’t use any other means of server state synchronisation, don’t use hardcoded sleep periods; replace them with an incremental backoff and verification in a loop. Sometimes you can’t predict whether a server state is updated properly in 500ms, 1000ms or 3000ms. Adding 5s waits everywhere may cause test suites to run veeery long.
  5. Be careful about leaking presences between cases. This applies to XMPP testing. Best practice is to generate fresh user accounts for every scenario.
  6. Some databases don’t support transactions so the new data may not be instantly available. For example, in the case of Riak (its Search function in particular) a delay between data insert and query is required.
  7. Sometimes creating a schema in a DB may fail for the first time due to timing issues, so implement a retry loop in a DB provisioning scripts. This also applies to Riak.
  8. Did I mention creating new user accounts for every test case? It actually applies not only to XMPP. With this practice, you won’t have to worry about possible leftovers of a user’s state.

Honorable mentions

ElasticSearch backend for Message Archive

Almost every MongooseIM module supports more than one type of backend. Usually it’s Mnesia, RDBMS and sometimes Riak. Message Archive Management is a noteworthy exception, as we’ve implemented RDBMS, Riak and Cassandra support for this module. Or “modules” actually, as it consists of over 30 Erlang files already.

It is a very special feature as it processes a vast amount of data and sometimes executes expensive queries. In order to ensure performance and match a project’s architecture, wide range of supported DB backends is essential.

It is our pleasure to announce that yet another backend has joined the family: ElasticSearch.

OTP 21 support

OTP 21.0 has been released ~1 month ago and we’ve added support for this version less than a week after! This is great news for all projects sticking to the most recent Erlang technology as pioneers in BEAM world. The new platform version brought not only improvements in regards to performance but also some incompatibilities that we’ve resolved, so MongooseIM still remains at a technological peak.

As a tradeoff, we’ve dropped official support for OTP 18.x. It should still be possible to compile 3.1 with this version with some minor code modifications, but we’re definitely moving forward. It has allowed us to get rid of non-typed maps specifications as an example. As a reminder, bear in mind, that MongooseIM always supports two most recent, stable OTP branches (currently these are 20.x and 19.x and one being under an active development, 21.x).

Jingle/SIP tutorial

SIP is a common choice for VoIP applications but certain XMPP features may be a very good match for such software. MongooseIM is able to liaise between these two worlds and now it’s easier than ever with significantly extended documentation (compared to the level in 3.0) and a tutorial on mod_jingle_sip usage.

Worker pool unification

Every developer writes a custom worker pool at some point of their career. Everyone. Certain MongooseIM components (the ones that use connection pools) were created with different preferred library in mind. As a result, we’ve ended up with many kinds of worker pools: cuesport, worker_pool and poolboy. It wasn’t only a matter of maintenance difficulty, but performance as well. As an example, cuesport supports only a simple round-robin job assignment algorithm, which is not optimal in every case. It also lacks inspection of any kind.

Given our experienced gathered over the years, we’ve selected worker_pool as our primary library. It is very flexible and exposes a dedicated stats API. It was originally created by Inaka, however it is actively maintained by Erlang Solutions at this present time.

For now, the changes are purely internal. Some projects may observe better performance but the primary goal was to prepare for a second round of unification. Stay tuned for more details in near future.

Changelog

Please feel free to read the detailed changelog. Here, you can find a full list of source code changes and useful links.

Contributors Special thanks to our contributors: @SamuelNichols @Beisenbek @GalaxyGorilla!

Test our work on MongooseIM 3.1 and share your feedback

Help us improve the MongooseIM platform:

  1. Star our repo: esl/MongooseIM
  2. Report issues: esl/MongooseIM/issues
  3. Share your thoughts via Twitter: @MongooseIM
  4. Download Docker image with new release
  5. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
  6. Check out our MongooseIM product page for more information on the MongooseIM platform.

July 23 2018

Peter Saint-Andre: Intensive Reading

As previously mentioned, I have in mind to write a novel about the ancient Greek philosopher Pyrrho and his journey to central Asia and northwestern India with the army of Alexander the Great. As part of my research I'm reading a great deal of philosophy, history, and fiction. It's been fascinating to return to the reading of fiction after a lapse of many years - I feel like I have fresh eyes for everything. My readings in the genre of philosophical fiction have ranged all over the map - during a business trip last week I devoured The Name of the Rose by Umberto Eco, before that I re-read The Left Hand of Darkness by Ursula K. Le Guin, and I'm currently reading both The Picture of Dorian Gray by Oscar Wilde and The Awakening by Kate Chopin. Alongside these works of fiction I'm reading books of philosophical scholarship (the latest were a full-length study of Pyrrho and an exploration of epistemology after Protagoras) and I'm starting to explore the historical background to the massive cultural change from classical to hellenistic Greece. Umberto Eco's book in particular has given me a new ambition of combining the story of Pyrrho's intellectual development with a mystery novel about the still unexplained death of Alexander the Great! It turns out the Pyrrho was probably not quite the radical skeptic he's usually taken to be (although he did believe that the entities we encounter in reality are indefinite and thus hard to pin down in many ways). Combine this with political intrigue on an imperial scale and perhaps regicide (indeed, there's an ancient legend that Aristotle himself was somehow involved in the assassination of his former pupil Alexander, perhaps in revenge for Alexander's execution of Aristotle's nephew Kallisthenes), and we might have the makings of large-scale novel that hinges in part on the validity of the law of non-contradiction! This is all quite speculative at this point because I have over 200 more books to read during the research phase, and I would also need to intensively study how the better authors construct their novels (an art of which I am mostly ignorant). Although I'm not sure if I have a book this big in me, I'm enjoying the exploration of possibilities. And all this reading has kept me from blogging, thus my absence here. Hopefully the results will be worth waiting for......

July 19 2018

Paul Schaub: Summer of Code: First PGPainless Release!

I’m very happy and proud to announce the first alpha release of PGPainless!

PGPainless 0.0.1-alpha1 is the first non-snapshot release and is available from maven central. It was an interesting experience to go through the process of creating a release and I’m looking forward to have many more releases in the future :)

The current release contains a workaround for the bug I described in an earlier blog post. The issue was, that bouncycastle wouldn’t mark the public sub keys of a secret key ring as sub keys, which results in loss of keys if the user tries to create a public key ring from the exported public keys. My workaround fixes the issue by iterating through all sub keys of an existing key ring and converting the key packages of subkeys to subkey packages. The code is also available as a gist.

Ironically I had some issues related to OpenPGP during the release process. Releases to maven central have to be signed with an OpenPGP key, so I created a dedicated signing key pair using GnuPG, which I wanted to put into a separate GPG key box. After creating the key, I exported it using

gpg --armor --export-secret-keys [key-id] > pgpainless-singing-key.asc

imported it into a dedicated key box using

gpg --no-default-keyring --keyring pgpainless.gpg --import pgpainless-signing-key.asc

and deleted the key from my old key box, as well as the .asc-file. But when I tried to sign my release, I got the error, that a secret key would be needed. After checking the key box, I noticed, that only a public key was present.

Unfortunately that key had already been published to the key servers and I have no way to revoke it, due to lack of a secret key. I have no idea, what exactly happened or how it could happen, but its too late to recover the key. So in the end I had to create a new OpenPGP key, which I now carefully backed up on a freshly bought USB stick which will be locked away for the event that I lose the copy on my work station. Better safe than sorry.

Happy Hacking!

July 17 2018

Christian Schudt: Babbler 0.8.0 released

After a longer period of development, Babbler 0.8.0 has just been released today.

The new major features are certainly

Further information about the release can be found in the release notes and in the changelog.

July 16 2018

ProcessOne: Announcing RTB, an open source real-time protocols benchmark tool

ProcessOne is excited today to announce the first release of RTB (Real-Time Benchmark), an open source benchmarking tool to stress-test real-time protocols. The aim of this project is to define an easy benchmark for XMPP and MQTT servers, with minimal configuration overhead.

Why

To compare server implementations and be sure they are ready for massive traffic, there is a need for a “golden standard” benchmarking tool that is easy to use and avoids ambiguity due to configuration or overhead – each test should have the same baseline and capabilities for the results to be comparable.

We believe RTB fits such a role, because it has sane defaults (gathered from statistics of real world servers) and is able to cover all the test features defined in the XMPP Compliance Suite 2018.

Status

RTB is in early stages of development: currently, only XMPP protocol is implemented, and support for Multi-User Chat (MUC) and Personal Eventing Protocol (PEP) is lacking.

However, “sane” defaults and what should be considered a “golden standard” for this benchmark is open for discussion with the XMPP community – we welcome issues and pull requests. The tool has already been battle-tested: we used RTB to stress test our ejabberd SaaS deployments with more than 2 000 000 concurrent connections.

RTB includes a script to populate your server’s database with user accounts and rosters, so it’s easy to create test pools. When you start a test with RTB, you can monitor a local website on port 8080 with live graphs like the one below:

Requirements

We want RTB to be straightforward and easy to deploy. Currently, only Unix-based operating systems are supported. To compile, you just need gcc, make, Erlang, expat, yaml, openssl, zlib and gnuplot. There’s a default YAML configuration file to get you started and only 5 mandatory parameters before you can lanuch your first benchmark.

RTB is available on GitHub under Apache License 2.0 and we are waiting for your feedback!

Paul Schaub: Summer of Code: Plan for the grand finale

I passed the second evaluation phase :) Now begins the final spurt, as the last month of GSoC has begun. My main goal can be summarized as follows: Get everything merged!

To get that done, I have to polish up my smack-openpgp branch which has grown to a size of 7000 loc. There are still some minor quirks, but Florian recommended to focus on the big picture instead of spending too much time on small details and edge cases.

I also have to release pgpainless to maven central and establish some kind of release cycle. It will be a future challenge for me personally to synchronize the releases of smack-openpgp and pgpainless.

But now enough talking, I have to get to work :)

Happy Hacking!

July 13 2018

Paul Schaub: Summer of Code: Second evaluation phase

Quite some time has passed since I bothered you with my last post :) A lot has happened since, I have been making steady process in both smack-openpgp, as well as pgpainless.

One big step that I took was to get rid of smack-openpgp-bouncycastle, which now has been merged into smack-openpgp. Having modular code may be worthwhile, however it poses some big challenges. The biggest problem with having smack-openpgp not depend on smack-openpgp-bouncycastle was, that I could not use classes that represent encryption keys directly in smack-openpgp. Instead I had to create interfaces that encapsule functionality and call those in order to get stuff done from inside smack-openpgp. Last week me and flow decided that it would make my job a lot easier if we just got rid of smack-openpgp-bouncycastle by merging the two modules. In case there will be another implementation at some point, the code would still be modular enough to allow extension by overriding classes and methods.

Now smack-openpgp depends on pgpainless directly, which means that I don’t have to create duplicate code to get bundled information from pgpainless to smack-openpgp for instance. This change gave me a huge performance boost in the development process, as it makes the next steps much more clear for me due to less abstraction.

I rewrote the whole storage backend of smack-openpgp, keeping everything as modular as possible. Now there are 3 different store types. One store is responsible for keys, another one for metadata and a third one for trust decisions. For all of those I created a file-based implementation which just writes information to files. An implementor can for example chose to write information to a database instead. For all those store classes I wrote a parametrized junit test, meaning new implementations can easily be tested by simply inserting an instance of the new store into an array.

Unfortunately I stumbled across yet another bug in bouncycastle, which makes it necessary to implement a workaround in my project until a patched version of bouncycastle is released.
The issue was, that a key ring which consists of a master key and some subkeys was not exported correctly. The subkeys would be exported as normal keys, which caused the constructor of the key ring to skip those, as it expected sub keys, not normal keys. That lead to the subkeys getting lost, which caused smack-openpgp to be unable to encrypt messages for contacts which use a master key and subkeys for OpenPGP.

This bug has been fixed pretty fast by the bouncycastle team and the minimal test I created to illustrate my problem has been incorporated into bouncycastle. Thank you :)

Currently I’m working on a workaround for the bug in smack-openpgp, but that work is already working. Next I will polish up my test client and do some more field testing to iron out all the edge cases I probably overlooked :)

Happy Hacking!

July 05 2018

Jérôme Poisson: Salut à Toi 0.7 alpha: contributors, take your keyboards!

It's my pleasure to announce the release of the first alpha version of Salut à Toi 0.7.

This is the occasion on one hand to show the state, advancement and direction of the project, and on the other hand to launch a call for contributions.

In this article I will show you a part of Salut à Toi capabilities, and explain briefly how to use them.

You'll find a link to an .apk file to test Cagou on Android in the "Test it!" section.

As a reminder, Salut à Toi is a collaborattive ecosystem, which is multi-frontends (web, desktop, mobile devices, terminal, command line), multi-platforms, and multipurpose. It let you handle your communications, organise yourself, share your files, etc. It is also, and above all, a militant project which think about the impact of new technologies on our lives, and which tries to stimulate public debate.

We'll concentrate on the web and desktop/mobile frontends, because those are the ones which will most probably be of interest to the mass.

Web frontend (named "Libervia")

Libervia has seen major changes, with a new web framework.

The idea, which has been explained in a blog post is to offer base components (account authentification, permissions, comments, etc.) ready-to-use and decentralised thanks to XMPP. This way we easily build website which are automatically decentralised and federated.

Please note that everything is prepared for traduction, but for now only English is available. Help would be much appreciated to improve on that!

Let's check the available features:

Chat

screenshot of a "chat" discussion in Libervia

Let's start with the most obvious, there is an instant messaging page. To use it, you just have to click on "chat", et to log in if it's not already done.

You'll land on a panel with the rooms in your bookmarks, you just have to click on one of them, or to specify an identifier (jid) of a new room/contact.
It is for now quite basic, but still you can enter "/help" to check textual commands.

Blog

my own blog, powered by Libervia

One of the flagship features of SàT, really rare in XMPP world (the only actively developed other client implenting it being Movim), is the blog (which is then decentralised).

A click on the menu will bring you to a "discover" page where you'll find your contacts with a blog. Click on one of them to read, or specify an identifier to read a blog which is not in your contacts.

Forums

XMPP powered decentralised forums

Unique feature in XMPP world, SàT propose forums, allowing to organise discussions in categories. Because forums are under the hood the same thing as blogs, it is possible to use a blog as forum and vice versa.

The addition of new forums is done by an administrator using command line tool "jp" (see below).

Events

personal list of events

Other feature not seen elsewhere in XMPP (and in decentralised software?), SàT allows you to create events, and of course to invite people.

As with other features, the page is thought to be easy to use: you have a button "create an event" which bring you to a form. A name, location and date later (and optionally a picture to associate), and your event is created. You'll then land to an admin page where you'll have the possibility to invite people.

Events are private for now, only your guests and you have access to it, but public events are also planned.

a guest is answering to an event RSVP

Please note that it's possible (but not yet from the web interface) to add arbitrary fields to your events, for instance to let your guest specify what they'll bring to a picnic.

File sharing

sharing file from browser

Here we are talking about a feature useful daily, and in many circonstances. Salut à Toi has long been able to send files, or to upload them on your server, but we are talking here about sharing one or many directories, or to upload in an advanced way on a serveur component.

All of this is explained in a recent blog post, but let me add 2 use cases to understand well:

  • you are home, you've just taken a video and some photos on your phone that you want to get on your computer. You have installed Cagou, the desktop/Android frontend of SàT on your device and already shared your photo directory. From your desktop computer, in the "file sharing" mode you see your device and you can directly download the video and the photos from there. The connection will be done directly on your local network, from device to device, and your files will not go throught a external server located whoever knows where.

  • you have files that you want to access at any time from anywhere (pictures or working documents for instance). You put them in a virtual directory on your file sharing component (a SàT instance too) and you can access it as simply as for the previous example.

Note that while with Cagou the connection is done directly when possible, it's not the case for Libervia which will first download the file on the server, before taking it throught HTTPS. WebRTC will be needed for the direct connection, and that's planned for 0.8 release.

Permission are handled easily: you specify who can access your files in a couple of clicks.

Files are transfered in clear for now (unencrypted), but this should not be the case anymore once finale release is here.

Photos albums

a photos album with a comment

Based on previous feature, a photos album mode display a directory containing only photos with adapted view, an with comments.

Permissions are really useful here, allowing to let, for instance, only your family see an album.

It's not yet possible to access an album from the interface of Libervia, to test it you'll have to change the URL by hand from the file handling page (by replacing files/list by photos/album).

Tickets

SàT tickets, first ones have been imported from Bugzilla

Developed from our own needs, SàT now handles tickets in pretty much the same way as a code forge. This is classical issue tracking, bugs report, tasks to do, etc.

Fields are free, it's really easy to use this feature for any kind of list: TO-DO tasks, but also, for instance, shopping lists.

Merge requests

a major patch is being reviewed

In the same spirit as tickets, SàT also handle "merge requests". Developed and used primarly for SàT itself, this system has the benefit or not being tied to a specific tool (like Git or Mercurial). Il is currently used with Mercurial, but it is possible to use it with whatever we want, and I'm not only talking about software code here.

Those 2 features means that SàT is a decentralised code forge. Even if it's still relatively basic, it has the benefit of being easy to use, flexible and adaptable. Furthermore there are only a few decentralised code forge at the moment (we can probably count Fossil, maybe some others).

Once again, contributions would accelerate things and would permit to have a very powerful tool.

Application

the dynamic part, which in under minimal maintenance and will be re-written for 0.8

To conclude this small tour of Libervia, we have to mention the application which is an entirely dynamic page, more elaborated but also more complicated to use (one of the main base version goal is to be easy to use).

Libervia web application allows to use widgets that you can drop next to each other, to do end to end encryption (OTR), or to publish blog post in simple or WYSIWYG mode, with the syntax you like (Markdown for instance).

It's the historic application which, for technical reasons, is now under minimal maintainance and have a few issues (like the contacts which can appear 2 times as seen in the screenshot above). It will be re-written for SàT 0.8, but it's let in place in this release for people who want to use it.

Cagou (desktop/Android)

Promised after the success of our crowfunding campaign, which allowed us to get a bit more than 3000 €, Cagou is our new frontend for desktop and Android devices. Its goals are to be modern, easy to use, and powerful.

Its original UI is inspired from Blender, and let split the screen to show several widgets at once. This UI has been thought in a way that it can be used even without knowing this split capacity, while being straightforward to use once the feature has been discovered.

To change mode, you just have to click on the top left icon.

For now, 5 modes are available: chat, contacts, file sharing, widget selector, and settings.

On the very top, you have a menu (only on desktop), and a notification zone, where messages appear for a few seconds and are visible entirely when clicking on the head of the left Cagou. On the other side, an other Cagou head can appear when you get notifications needing a user interaction (new contact for instance, or authentification request – see below –). This way, no popup will steal your focus while you are writing a message or doing something else.

Split

It is possible de split the screen as much as you want by clicking on the 3 dots area (on top and left borders), and then moving your mouse/finger. If the bar is red, this means that you'll close the area.

Here is a small animation to make things more clear:

demo of Cagou's widget split ability

You can organise the way you want, and monitor several things at the same time. In the future, it should be possible to keep handy widgets organisations so you can re-use them easily.

Chat

Cagou "chat" mode screenshot

The chat should be relatively intuitive. The top area show the current room or the identifier (JID) of the person you're talking to. It is possible to enter an other identifier manually to change room. For instance you can enter sat@chat.jabberfr.org to come on Salut à Toi room.

Still on the top area, the "padlock" icon on the right (only visible in 1/1 discussions) allows you to encrypt the conversation end to end, with OTR only for now (see below).

If you have several discussions at the same time, you can switch from one to the other by swiping your finger horizontally. Only discussions not already visible somewhere on the screen will be available, and for now they are in alphanumeric order (most certainly recent discussions will be available first when stable release will be out).

Finally, in the bottom area, in addition to the text input, you can see a + button with a circle which allows you to open a media transfer panel. On the desktop for now you can only select a file from your file-system, but on Android you can select/take a picture/video, a file or record an audio message. In every cases you can choose if you want to upload the file on your server, or send it directly to your contact, without transiting by the server.

file sharing

The principle has already been explained for Libervia, here is an animation showing how to share a directory with Cagou:

sharing a directory from Cagou

As you can see, you just have to do a long click/touch on the directory or file to share, and to select people which will have access to it.

You can click on "share local files" button, in the header zone, to change mode and see the device which are sharing. You'll then see – as with Libervia – 3 areas: the first one with sharing component, if any, the second one with your own devices (your phone or desk computer for instance), and finally the devices of your contacts. Once again, the input area let your filter devices to show.

see devices which can share files

Third party website authentification

SàT is a project which push the use of XMPP in every directions, and this include authentifying on third party websites. The idea is to be able to authenticate yourself without having to create an account on any website just by giving your JID.

Here is a video demo, thanks to the demo website made by jnanar, the screen on the right is a phone (displayed thanks to scrcpy):

super easy authentication with XMPP

You can implement this in your own website either by using Chteufleur component (Chteufleur is actually the original implementer of the feature in SàT, many thanks), either by using a server where it's already available (it's for instance the case on jabberfr, thanks to the admins of this server).

Please note that once it's in place, in your website you just have to do a single HTTPS request to validate authentication.

Contacts

No need to take much time on this mode, but it's good to know that you can add a new contact there (with the top button), or delete an existing one with a long click/touch.

A simple click will launch the chat mode with the selected contact.

Command line (JP)

There are far too many new things in jp to describe everything here.

Some useful informations anyway:

  • in jp we always put commands first, then arguments (jp [command] [subcommand] --arg_1 --arg_2 --etc)
  • jp [command] --help tells you how to use a command
  • jp shell let you use a REPL shell
  • it is possible to have high level and low level commands. For instance with the blog, you can have blog posts in a nice way by using jp blog get -s bloger@somewhere.tld -O fancy, or the raw XML with jp pubsub get -s bloger@somewhere.tld -n urn:xmpp:microblog:0
  • many commands have an output mechanism with --output (or -O). For instance jp blog get -s bloger@somewhere.tld -O json let you find metadata of a blog feed in JSON
  • jp can generate static websites thanks to --output template
  • when the website generator handle it, you can directly use the blog URL to edit it. For instance, I can edit one of my last posts by using jp blog edit -u https://www.goffi.org/b/khii3yEKWDhmNYSDt5GGrY/vers-forge-decentralisee-basee-xmpp where https://www.goffi.org/b/khii3yEKWDhmNYSDt5GGrY/vers-forge-decentralisee-basee-xmpp is the URL of the HTML generated page. I then have my text editor showing the code of my page on one side (in my favorite syntax) and the metadata on the other. Note the this works with webpage generated by Libervia, but also by Movim.

A small demo of the shell to read a blog:

checking a blog from shell (jp)

Test it!

You can test all this, the installation instructions are available (in English only, we are lacking resources to translate) on the wiki by clicking here.

SàT (backend + jp (CLI) + Primitivus (TUI)) is available in Debian experimental (thanks to Robotux, Naha, Debacle and other packagers).

SàT and its frontends (jp, Primitivus, Cagou, Libervia) are available on Arch Linux (thanks to jnanar and Link Mauve)

We got confirmation that SàT and Cagou are working on Mac OS X (thanks to xma), we would like help of people to also test it on Windows, BSD*, or anything else.

You can test on Android by clicking on this link from your device (installation from external resources must be activated from settings).

Note that this is an alpha version, that means:

  • it crashes, the code is not stable
  • all features are not finished, or even here
  • it's not optimised yet
  • it's not polished

Oh, and while we are talking about features not yet implemented, there are strong possibilities that end to end encryption with OMEMO will be implemented before final release, thanks notably to the new Python module available (thanks to Syndace, the author). End to end encryption is already available for years in SàT, but only with OTR so far.

Participate!

A last animation to show you the automatic language detection, an experimental feature:

language detection, then messages filtering

Let's stop here for the features, but we have not seen all of them.

As you can see, there's a lot to play with, and there is enough for every taste in this project.

And that's good, with summer holidays you'll have more time, a perfect occasion to contribute to a unique project, ethics and militant.

Do not hesitate to come on SàT XMPP room (sat@chat.jabberfr.org, also available without account, from the web).

If you are lacking time (picnic on the beach is nice too), support on Liberapay and/or adhering to the association is also helping.

Try to spread the word around you, our project lack visibility and needs you. If you are willing to host an instance, you can start to test (in a "experimental/not stable" part of your website), and give us feedbacks. We are also looking for people helping to package in every place where it's possible.

Thanks in advance, and see you soon!

July 02 2018

Paul Schaub: Summer of Code: Checkstyle to the rescue!

Today I added some checkstyle rules to PGPainless.Checkstyle is a gradle plugin, which checks the source code for style violations.

Some say, strict checkstyle rules are unnecessary and that it is annoying to be held back from pushing a commit to the master branch only to fix “style issues” for half an hour. I must say, in the beginning I thought the same way. I was annoyed thinking “why does it matter, if a line comment ends with a period or not?” But after being forced to give it a try when I first became a contributor to the Smack project, I became a fan of it. In the beginning I had to often recommit my changes because they broke the checkstyle rules. For example I often forgot to leave spaces between mathematical operators. I would write “i = 5+5;” instead of “i = 5 + 5;”. But after some amount of time, I got less and less warnings.

I adopted most of the (honestly *very* strict) rules in Smacks rule set to my own coding style. I like how it automatically leads to cleaner, more uniform code (not that it is impossible to write garbage with it of course). For that reason, I decided to put those rules into place in PGPainless today (I only left one rule out, because who the hell cares about the alphabetical sorting of imports???).

At some point, PGPainless will be released as a maven artifact. In preparation for this historical event, I bought the domain pgpainless.org. For now it is just a forwarding to the PGPainless git repository, but I will likely setup a small website with documentation etc. at some point.

During my testing of Smacks OX implementation, I came across an interesting problem. When a user queries a PubSub node in Smack, Smack first does a disco#info query on that node to determine, whether it is a LeafNode or a CollectionNode. This normally works fine. However, it becomes more and more popular to make use of the PubSub access model ‘open’. The open access model makes a PubSub node accessible to entities (like other users) which are not in the contact list of the user. This enables the use of OMEMO in group chats, where not every participant is in your contact list for example.

The problem is that a server which allows access to open PubSub nodes, does not necessarily allow the disco#info query. The question is: Should disco#info queries on open PubSub nodes be allowed or not? An argument against it is, that it might allow “jid-harvesting”. An attacker might use disco#info queries on open PubSub nodes in order to determine, whether the user exists or not. This is a bad thing, because it allows spammers to collect the Jabber IDs of potential victims. On the other hand however, the attacker could simply do a direct PubSub query on the open node and the result would be the same. The benefit of allowing disco#info queries would be, that you can in fact determine the node type.
For now my mail to the standards mailing list remained unanswered, but I think that there should be a well defined expected behavior for this edge case.

For now I worked around the issue by using Javas reflections to access the LeafNode constructor directly, avoiding the disco#info query.

Other than that, I didn’t get a whole lot done this week. Unlike the demotivating week though, this time the reason was primarily exciting new hardware :D

Happy Hacking!

June 28 2018

The XMPP Standards Foundation: The XMPP Newsletter, 29 June 2018

Welcome to the XMPP newsletter.

If you have an article, tutorial or blog post you'd like us to include in the newsletter, please submit it on the XMPP wiki.

News

Github has been acquired by Microsoft, prompting some FOSS developers with long memories to reconsider whether they still want to host their projects there.

The Salut à Toi project is working on a decentralized and federated solution for issues and merge requests, based on XMPP, and Jérôme Poisson explains in this article their motivation for doing this work.

Every now and then a blog post comparing XMPP and Firebase appears. Generally these posts seem to be poorly disguised attempts at SEO and to promote application development by a particular devshop. This submission seems to follow the trend and is also relatively superficial. It is however interesting to note that they mention multiple times that it's difficult to set up an XMPP server. Perhaps something to take note of and to consider how we could ease setup of an XMPP-based solution.

Monal has received a lot of updates last months and now it also has initial OMEMO support. More work needs to be done, so don't expect to see this in the App Store before August.

Software releases

Servers

The XMPP server written in Go has a new release with support for Websocket (RFC-6395), XEP-0191 (Blocking Command), XEP-0012 (Last Activity) and XEP-0237 (Roster Versioning).

The new release includes improved robustness of external authentication backends, support for REST API custom headers and more.

This release includes the so-called "Inbox" feature which was also discussed at the XSF summit in FOSDEM. It is a list of all ongoing conversations, with excerpts of last messages and unread messages count, to help you gain an overview of what happened while you were gone. There is no XEP for this yet, but Erlang Solutions have promised to submit one.

Clients

  • Monal 3.0.2 has been released.

  • JSXC 3.4.0 released including support for location sharing and improved roster management in our internal Nextcloud backend.

Libraries

Chris Ballinger has released Version 4.0 of XMPPFramework, the XMPP library for iOS and macOS. It brings support for XEP-0048: Bookmarks and XEP-0359: Stanza Ids. It also has improved Swift support with optional Swift-only features.

ProcessOne: ejabberd 18.06

This ejabberd 18.06 release includes, after two months of development, a lot of bug-fixes and many improvements. There are also a few changes, so please read carefully the release notes.

Noticeable changes

Stop ejabberd initialization on invalid/unknown options

Since some time now, we are warning ejabberd admins in log file about incorrect configuration options. We feel that we are ready to make these hints strong requirements about config file validity.

Starting with ejabberd 18.06, ejabberd will not ignore unknown options and doesn’t allow to have options with malformed values. The rationale for this is to avoid unexpected behaviour during runtime, i.e. to conform to “fail early” approach. We thus hope to be helpful to newcomers by making sure they properly format their config files to get the expected behaviour.

Note that it’s safe to reload a configuration with potentially invalid and/or unknown options:
this will not halt ejabberd, but will only prevent the configuration from loading.


NOTE FOR PACKAGE BUILDERS
This new behaviour should be documented in the upgrade notes.

Improve robustness of external authentication backends

Now all external ports are attached to supervising processes and requests are balanced in round-robin manner until the pool is exhausted.

The commit also deprecates extauth_instances option and introduces extauth_pool_size option instead, with the default value of a number of logical processors (i.e. CPU cores).

Add new options for OOM watchdog

  • oom_watermark: 1..100
    Start OOM watchdog only when system memory usage exceeds this value in percents. When the usage drops below the value, OOM watchdog is stopped. The default is 80 (percents). Note that once OOM watchdog is started, it performs full garbage collection periodically: this can be seen as spikes in CPU utilization and drops in RAM usage. If your system is permanently above the watermark, it may cause significant CPU overhead.

  • oom_queue: positive integer
    Only trigger OOM killer when total amount of messages in all queues of all Erlang processes is above this value. The default is 10000. Note that this value only takes effect when oom_killer is set to true (this is the default). Otherwise, only a warning will be logged.

Add support for REST API custom headers

ext_api_headers can be defined as a single string. Headers are separated by comma. Definition MUST NOT contain spaces. Example “X-MyHead:test,X-Token:082748

Optimize HTTP requests memory usage

Due to historical reasons, ejabberd loads the whole file/data into the memory when serving an HTTP request. This is now improved:

  1. For GET requests ejabberd uses sendfile(2) if the underlying connection is HTTP and falls back to read/write loop with 64kb buffer for HTTPS connections. This type of requests are handled by mod_http_fileserver, mod_http_upload, ejabberd_captcha, etc
  2. POST requests are now limited to 20Mb and are fully downloaded into the memory for further processing (by ejabberd_web_admin, mod_bosh, etc)
  3. PUT requests (e.g. for mod_http_upload) are handled by read/write loop with 64kb buffer

Support SASL PLAIN by xmpp_stream_out

Also, SASL mechanisms chaining is now supported: if several mechanisms are supported and authentication fails, next mechanism in the list is picked, until the list is exhausted. In the case of a failure, the latest SASL failure reason is returned within handle_auth_failure/3 callback.

Drop support of mod_irc

If you need IRC support, we suggest to use biboumi. We will not maintain mod_irc anymore and moved it to ejabberd-contrib repository.

SQL schema change

As mod_irc has been obsoleted, we removed the related SQL table from the schema: irc_custom

Changes

Admin

  • Stop ejabberd initialization on invalid/unknown options
  • Add new options for OOM watchdog: oom_watermark and oom_queue
  • Add ability to modify version string
  • Add option ext_api_headers to define REST API custom headers
  • Fix Erlang limits in ejabberdctl.cfg.example to reflect current situation
  • Make trusted_proxied ejabberd_http option accept ip masks
  • Teach acl ip matching about ipv4 mapped ipv6 addresses
  • Removed watchdog_admins option from config, as has no effect anymore
  • Improve logging of external authentication failures
  • ejabberd_auth: Don’t use cache if the option is disabled
  • Make connected_users_info and user_sessions_info DB-agnostic

Core

  • Support SASL PLAIN by xmpp_stream_out
  • Add Resource Binding support to xmpp_stream_out
  • Improve robustness of external authentication backends
  • Don’t use ‘unsupported-version’ inside SM element
  • Generate SASL failures on unencrypted connections only for s2s
  • Fix reset_stream in websocket using pre-rfc protocol
  • Don’t crash in bosh when we receive request with RID < prev_rid
  • Get rid of all calls to jlib.erl module
  • Support IPv6 connections for PostgreSQL, MySQL and LDAP
  • Fix authentication for usernames containing uppercase characters
  • Optimize HTTP requests memory usage
  • PKIX: Just warn instead of ignore a certificate containing no domain names
  • PKIX: Don't replace valid certificates with invalid ones

Modules

  • Log modules startup
  • mod_disco: Advertise disco#info and disco#items features
  • mod_irc: is moved away from ejabberd repo to ejabberd-contrib
  • mod_mam: Don't replace existing stanza ID
  • HTTP upload: Generate HTTP Upload form using xdata codec
  • HTTP upload: Improve error formatting
  • HTTP upload: Return detailed error if HTTP upload is too large

MUC

  • Always display room's xdata in disco#info
  • Display muc#roomconfig_changesubject in room's disco#info
  • Render roomname, allowinvites and allowpm in room disco#info
  • Support for roomconfig_lang/roominfo_lang
  • mod_muc_sql: Fix export to SQL

Push

  • Omit summary for outgoing messages
  • Further improve handling of carbons
  • Also include sender/body for carbons
  • Include a static body text by default
  • keepalive: Increase default timeout to 3 days
  • SQL: Check 'max_user_sessions' limit

Feedback

As usual, the release is tagged in the Git source code repository on Github.

The source package and binary installers are available at ProcessOne.

If you suspect that you’ve found a bug, please search or fill a bug report on Github.

Monal IM: Monal has OMEMO

OMEMO conversations work in Monal. There is still a lot work to be done here to properly handle keys and make UI elements in OSX and iOS for all the the new interactions.  There is a lot that also needs to be tested before this is in ready for the App Store. Realistically, expect to start seeing this stuff in the binaries I release some point in  August.

I have tested this with gajim on linux and chat secure on iOS. Will have dust off an android device somewhere to try it with Conversations.

June 26 2018

Paul Schaub: Summer of Code: An (almost) three line fix to a three days problem

diff --git a/src/main/java/de/vanitasvitae/crypto/pgpainless/decryption_verification/DecryptionStreamFactory.java b/src/main/java/de/vanitasvitae/crypto/pgpainless/decryption_verification/DecryptionStreamFactory.java
index d651b1b..bca7ba4 100644
--- a/src/main/java/de/vanitasvitae/crypto/pgpainless/decryption_verification/DecryptionStreamFactory.java
+++ b/src/main/java/de/vanitasvitae/crypto/pgpainless/decryption_verification/DecryptionStreamFactory.java
@@ -157,15 +157,13 @@ public class DecryptionStreamFactory {
         PGPPrivateKey decryptionKey = null;
         PGPPublicKeyEncryptedData encryptedSessionKey = null;
         while (iterator.hasNext()) {
-            encryptedSessionKey = (PGPPublicKeyEncryptedData) iterator.next();
-            long keyId = encryptedSessionKey.getKeyID();
+            PGPPublicKeyEncryptedData encryptedData = (PGPPublicKeyEncryptedData) iterator.next();
+            long keyId = encryptedData.getKeyID();
 
             resultBuilder.addRecipientKeyId(keyId);
             LOGGER.log(LEVEL, "PGPEncryptedData is encrypted for key " + Long.toHexString(keyId));
-            if (decryptionKey != null) {
-                continue;
-            }
 
             PGPSecretKey secretKey = decryptionKeys.getSecretKey(keyId);
             if (secretKey != null) {
                 LOGGER.log(LEVEL, "Found respective secret key " + Long.toHexString(keyId));
+                encryptedSessionKey = encryptedData;
                 decryptionKey = secretKey.extractPrivateKey(decryptionKeyDecryptor.getDecryptor(keyId));
                 resultBuilder.setDecryptionKeyId(keyId);
             }
         }
         PublicKeyDataDecryptorFactory keyDecryptor = new BcPublicKeyDataDecryptorFactory(decryptionKey);

The above 3 deletions and 1 addition are the fix for a bug in my decryption routines, which took me 3 days to find. Lets examine the bug in more detail in order to understand what the code does, what went wrong, and why it took me so long to find it.

As I described in an earlier blog post, an encrypted OpenPGP message basically consists of the symmetrically encrypted body and a bunch of public key encrypted session keys. In Bouncycastle, the encrypted session keys are called PGPPublicKeyEncryptedData objects. The are encrypted using the asymmetric public key of a recipient and they contain the symmetric session key which was used to encrypt the actual message. So if Alice writes a message to Bob and Carla, the message will contain at least two PGPPublicKeyEncryptedData elements. One containing the session key encrypted for Bob, the other one for Carla.

In PGPainless, I iterate through all PGPPublicKeyEncryptedData objects of an incoming message in order to get a list of all recipients of the message. I do that because I can imagine it would be cool if Bob can see, that the message was also encrypted for Carla. Also during every iteration I see, if the current PGPPublicKeyEncryptedData object is encrypted for a key of which I do have the secret key available. If so, I extract the private key for later use.

Lets have an example:

Alice encrypts a message to both herself and Bob. Then she sends the message to Bob. Bob tries to decrypt the message.
PGPainless saves the first PGPPublicKeyEncryptedData object in the variable encryptedSessionKey.
This first encrypted session key has Alice’s key ID, so PGPainless adds the id to the list of recipient key ids and goes to the next iteration. Again it stores the encrypted data object in encryptedSessionKey.
This second encrypted session key has Bob’s key ID. PGPainless finds the respective private key, extracts is and saves it to the variable decryptionKey.

Now the iteration ends, as all PGPPublicKeyEncryptedData objects have been processed. Now the decryption can begin. Using the extracted private key, Bob decrypts the encryptedSessionKey to retrieve his session key with which he decrypts the message. Easy Peasy Lemon Squeezy – at least that is what I thought.

I wrote some JUnit tests that encrypt a message for one recipient and those tests work just fine. I used them for example to determine the average message length using different algorithms.

For some strange reason however, during integration testing I noticed that every now and then decryption would fail. I thought it had to do with some race conditions at first. I blamed JUnits @Before and @After annotations which I used to delete the key rings from disk after the test was done to fire too early. Coincidentally after I removed the annotations the test would work flawlessly, which encouraged me and seemed to confirm my hypothesis.

However, the more tests I did, the more confused I became, as I could not find the cause of the failing. Bouncycastle has debugging disabled for their library, so I could not follow the calculations step by step. This is done for security reasons I guess. Fortunately there is a debug version which I discovered today. Using this library, I can see, which lines are responsible for throwing exceptions and step through the execution in great detail.

The final hint however came from a forum post. So lets see what exactly went wrong. For that purpose lets assume the same scenario as above, but with a slight twist.

Again, Alice encrypts a message to herself and Bob, but this time she prepends Bobs session key first.
Bob, when decrypting the message, stores the first PGPPublicKeyEncryptedData object in encryptedSessionKey, notices that this is his own session key and extracts his respective private key into the variable decryptionKey.
In the next iteration (remember, Bob wants to know all recipients), he stores the next PGPPublicKeyEncryptedData object in encryptedSessionKey. This is Alices session key.

Now he proceeds with decrypting the encryptedSessionKey with his decryptionKey – and hell breaks lose. Why? Because at this point in time encryptedSessionKey actually contains Alices session key, not Bobs.

The tricky part about this bug is, that it only happens by chance. I’m not sure in which order session keys are prepended to the message by Bouncycastle, but in certain cases this caused my code to fail – while in certain cases it did not. One benefit that OpenPGP has over OMEMO is, that you can write better tests. In OMEMO keys always change when the ratchet advances, so you cannot really test if certain messages decrypt. At least it is much easier using OpenPGP. But this bug told me a lesson, that you have to be very very careful with your JUnit tests. At some point I replaced randomly generated key pairs with some fixed keys to get more reliable test results and to be able to confirm the result using GnuPG. Ironically I was very lucky that as a result my test would reproduce the second scenario above. If instead it would have produced the first scenario, it would have taken me much much longer to discover the cause of the issue.

Fighting this bug took me 3 days. Luckily I didn’t spend 100% of my time bug hunting. I also wrote another integration test, which covers one very cool feature of OpenPGP for XMPP.

Secret Key Synchronization

In OX it is possible to store the secret OpenPGP key in a private PubSub node. That way the key can be easily transported to other devices, so you can read your encrypted messages in a web browser for example. This is one huge benefit over OMEMO, where you can only read messages received *after* you began using the new device.

During my testing, I also found out, that ejabberd despite announcing support for alternative PubSub access models, does not properly handle some requests.
For normal messaging, OX recommends to publish the public keys of users in a PubSub node configured with the ‘open’ access model. That allows users to access your public key, even if they are not in your roster.

Smack however does a disco info query on a PubSub node prior to fetching it in order to get some metadata about it. It turns out that ejabberd does return an error for such a request, stating that you have to be subscribed to the owner of the node in order to fetch information about it. I’m pretty sure though, that this bug will be fixed soon :)

Happy Hacking!

June 20 2018

ProcessOne: Real-time Stack Issue #11

ProcessOne curates two monthly newsletters – tech-focused Real-time Stack and business-focused Real-time Enterprise. Here are the articles concerning tech aspects of real-time development we found interesting in Issue #11. To receive this newsletter straight in your inbox on the day it’s published, subscribe here.

How to Build a Private One to One Chat App from Scratch?

In this article, the development of one to one chat app from scratch is discussed in detail. The technical stack involved in WhatsApp like chat app are Erlang (Language), Ejabberd (Framework), XMPP protocol, MySQL (Database). The following steps will help you to develop a one to one chat application in an hassle-free manner.

Deploy Bulletproof Embedded Software in Elixir with Nerves

Nerves defines an entirely new way to build embedded systems using Elixir. It is specifically designed for embedded systems, not desktop or server systems. It consists of a minimal Buildroot-derived Linux that boots directly to the BEAM VM, ready-to-go library of Elixir modules to get you up and running quickly and powerful command-line tools.

Connecting an Elixir Node to the Bitcoin Network

Pete Corey writes: “Since I first started diving into the world of Bitcoin development, I’ve wanted to build a simple node that connects to the network. The Elixir programming language gives us some fantastic tools to implement a server in the peer-to-peer network. Let’s see how far they can take us!”

Riot: A Distributed IRC and VOIP Client and Home Server

Riot is a free and open source decentralized instant messaging application that can be considered an alternative to Slack. This article takes a look at features of Riot, installation procedure and usage.

Elixir’s Phoenix Powering a Real-time Web UI

A problem WallarooLabs needed to solve early on was deciding on the tooling that would power our metrics monitoring system. They needed a monitoring solution to provide real-time updates on the several steps a data message may take along the way within a Wallaroo application. This post takes a deeper dive into the monitoring problem and how Phoenix and Elixir helped solve specific issues.

Fortnite: Postmortem of Service Outage at 3.4M CCU

Epic Games posted a postmortem on, among other things, their XMPP outage. They had a situation while mitigating a known instability problem that resulted in overloading a downstream system component and effectively paralyzing presence flow. Without presence, a user who is a friend cannot see that a player is online, breaking most of Fortnite social features including the ability to form parties.

June 19 2018

Paul Schaub: Summer of Code: The demotivating week

I guess in anybodies project, there is one week that stands out from the others by being way less productive than the rest. I just had that week.

I had to take one day off on Friday due to circulation problems after a visit at the doctor (syringes suck!), so I had the joy of an extended weekend. On top that, I was not at home that time, so I didn’t write any code during these days.

At least I got some coding done last week. Yesterday I spent the whole day scratching my head about an error that I got when decrypting a message in Smack. Strangely that error did not happen in my pgpainless tests. Today I finally found the cause of the issue and a way to work around it. Turns out, somewhere between key generation and key loading from persistent storage, something goes wrong. If I run my test with fresh keys, everything works fine while if I run it after loading the keys from disk, I get an error. It will be fun working out what exactly is going wrong. My breakpoint-debugging skills are getting better, although I still often seem to skip over important code points during debugging.

My ongoing efforts of porting the Smack OX code over from using bouncy-gpg to pgpainless are still progressing slowly, but steady. Today I sent and received a message successfully, although the bug I mentioned earlier is still present. As I said, its just a matter of time until I find it.

Apart from that, I created another very small pull request against the Bouncycastle repository. The patch just fixes a log message which irritated me. The message stated, that some data could not be encrypted, while in fact date is being decrypted. Another patch I created earlier has been merged \o/.

There is some really good news:
Smack 4.4.0-alpha1 has been released! This version contains my updated OMEMO API, which I have been working on since at least half a year.

This week I will continue to integrate pgpainless into Smack. There is also still a significant lack of JUnit tests in both projects. One issue I have is, that during my project I often have to deal with objects, that bundle information together. Those data structures are needed in smack-openpgp, smack-openpgp-bouncycastle, as well as in pgpainless. Since smack-openpgp and pgpainless do not depend on one another, I need to write duplicate code to provide all modules with classes that offer the needed functionality. This is a real bummer and creates a lot of ugly boilerplate code.

I could theoretically create another module which bundles those structures together, but that is probably overkill.

On the bright side of things, I passed the first evaluation phase, so I got a ton of motivation for the coming days :)

Happy Hacking!

June 17 2018

Ignite Realtime Blog: Smack 4.3.0-rc1 and 4.4.0-alpha1 released

@flow wrote:

The Smack developer community is proud to announce the availability of the first release candidate of Smack 4.3. Users of Smack are encouraged switch to the new 4.3 release family of Smack. The Smack 4.3 API is considered frozen and the API changes between 4.2 and 4.3 are not as significant compared to the changes between Smack 4.1 and 4.2. More information can be found in the Readme of Smack 4.3 (please note that the Readme is work in progress).

Together with the 4.3.0-rc1 release, we have also published the first alpha of Smack 4.4, which includes the updated and improved OMEMO API. Credits for this go to Paul.

As always, all the release artifacts are available on Maven Central.

Posts: 1

Participants: 1

Read full topic

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl