You are here

Agreguesi i feed

Ick version 0.53 released: CI engine

Planet Debian - Mër, 18/07/2018 - 5:17md

I have just made a new release of ick, my CI system. The new version number is 0.53, and a summary of the changes is below. The source code is pushed to my git server (, and Debian packages to my APT repository ( See for instructions.

See the website for more information:

A notable change from previous releases should be invisible to users: the release is built by ick2 itself, instead of my old mostly-manual CI script. This means I can abandon the old script and live in a brave, new world with tea, frozen-bubble, and deep meaningful relationships with good people.

Version 0.53, released 2018-07-18
  • Notification mails now include controller URL, so it's easy to see which ick instance they come from. They also include the exit code (assuming the notification itself doesn't fail), and a clear SUCCESS or FAILURE in the subject.

  • Icktool shows a more humane error message if getting a token fails, instead of a Python stack trace.

  • Icktool will now give a more humane error message if user triggers the build of a project that doesn't exist, instead of a Python stack trace.

  • Icktool now looks for credentials using both the controller URL, and the authentication URL.

  • Icktool can now download artifacts from the artifact store, with the new get-artifact subcomand.

  • The archive: workspace action now takes an optional globs field, which is a list of Unix filename globs, for what to include in the artifact. Also, optionally the field name_from can be used to specify the name of a project parameter, which contains the name of the artifact. The default is the artifact_name parameter.

  • A Code of Conduct has been added to the ick project. has the canonical copy.

Lars Wirzenius' blog englishfeed

Facebook is overly optimistic with respect to Cambridge Analytica data scope

Planet Debian - Mar, 17/07/2018 - 11:20md

Facebook is too optimistic when it comes to Cambridge Analytica extends.

Sorry for this post on a fairly old topic. I just did not get around to write this up.

Several media outlets (e.g., Bloomberg) ran the story that Facebook privacy policy director Stephen Satterfield claimed that “European’s data” may not have been accessed by Cambridge Analytica in an EU hearing.

This claim is nonsense. It is almost a lie - except that he used the weasel word “may”.

For fairly trivial reasons, you can be sure that the data of at least some European’s data has been accessed. Largely because it’s pretty much impossible to perfectly separate U.S. and EU users. People move. People use Proxies. People use wrong locations. People forget to update their location. Location does not imply residency nor citizenship. People may have multiple nationalities. On Facebook, people may make up all of this, too.

Even if Dr. Aleksandr Kogan did try his best to provide only U.S. users to Cambridge Analytica, there ought to be some mistakes. Even if he only provided the data of users he could map to U.S. voter records, there likely is someone in there that has both U.S. and EU citizenship. Or that became a EU citizen since.

Because they shared the data of 87 million people. According to some numbers I found, there are around 70,000 people with U.S. and German citizenship. That is “just” a tiny 0.02% of U.S. citizens. Since Facebook users are younger than average, and in particular kids will often have both citizenships if their parents have different nationalities, we can expect the rate to be higher than that. If you now draw 87 million random samples, the chance of not having at least one of these U.S.-EU-citizens in your sample is effectively 0. This does not even take other EU nationalities into account yet.

Already a random sample of 100,000 U.S. citizens will with very high probability contain at least one E.U. citizen (in fact, at least one German citizen, because I didn’t include any other numbers but the 70,000 above). In 87 million, you likely have even several accounts created for a cat.

Says math.

To anyone trained in statistics, this should be obvious version of the birthday paradoxon.

So yes, I bet that at least one EU citizen was affected.

Just because the data is too big (and too unreliable) to be able to rule this out.

Apparently, neither the U.S. nor Germany (or the EU) even have reliable numbers on how many people have multiple nationalities. So do not trust Facebook (or Kogan’s) data to be better here…

Erich Schubert Techblogging

Taiwan Travel Blog - Day 6 & 7

Planet Debian - Mar, 17/07/2018 - 8:30md

This is the fifth entry of my Taiwan Travel blog series! You can find my previous entries here:

I wasn't sure if people were enjoying my travel blog or if I was spamming planet.d.o with pictures of random mountain paths, but several people told me they liked it. Thanks for the feedback!

I've been busy in the last few days so for convenience's sake, I'll merge together what I did on the 14th and the 15th.

From mountain to sea

I left late in the morning on the 14th from the Taroko national park were I was staying to move to Hualien. Taroko was beautiful, but there is only so much to do there and I think I did most of it.

The bike ride was easier than I thought it would be. Taroko is in the mountains so I was travelling on a downward slope pretty much the whole way. There wasn't a dedicated bike path, but the road I took (n° 193) had a speed limit of 40km/h. The view was beautiful, as this road follows the shoreline all the way to Hualien.

I guess I must have been quite a sight for the locals: a foreigner riding a bicycle a few sizes to small for him on a small country road with a large bag and hiking boots strapped behind on the rack.

Fun times! I also caught a bad sunburn, as it seems the sun is stronger here than at home :(


After more than a week of travel, Hualien was the first large city in Taiwan I visited. Although its the largest city on the east coast, Hualien only has 100'000 inhabitants. That's a manageable size for me.

I arrived in Hualien in the beginning of the afternoon and after having checked-in at my hotel, I decided to go out for the night to have a meal and enjoy the city. I'm really happy I bought a bicycle, as it makes moving around so much easier than walking!

Following Andrew's advice, I first stopped at Danji's Bianshi shop (戴记扁食) to enjoy one of the best Bianshi I ever had. From what I understood from the pictures on the wall, this shop has been there for a least a few decades and only sells one dish: Bianshi. For those of you not familiar with Bianshi, it's a dumpling soup similar to Wonton soup, but where celery plays the leading role in flavouring the clear chicken broth. Hmmmmmm.

At the night market I got to eat a ton of food once again, from donuts filled with red bean paste to octopus takoyaki. It's also the harvest season here so I enjoyed many fresh juices like watermelon (the first time I had some juiced) and later on white jade bitter melon (白玉苦瓜), deliciously sweet and bitter.

And the Lord said ‘This bike shop shall be closed on Sunday’

On the 15th, I went out to try to get my bike fixed. I wanted to buy a longer saddle post and the bottom bracket on my bike was loose and I wanted to have it fixed.

Since the bike I bought is still under warranty, I had the great idea to cycle across town to the Giant store to see what could be done. Of course, I didn't check if the store was open and ended up realising it was Sunday when I saw the sign on the door.

Some say bad things happen for a reason so I comprised, bought fruits from the fruit stall across the street and decided to go watch the waves on the beach. The mango I ate was delicious and it had been a really long time since I ate fresh litchis.

I came back to my hotel in the middle of the afternoon, once again thinking I would have a quiet night listening to podcasts while working on some DebConf stuff when aLiao (Andrew's friend who owns a recording studio in Hualien) reached out to me to ask if I had plans for the night.

I ended up at his studio listening to him and a bunch of his friends jamming. We later went to a seafood restaurant on the edge of the night market called The Tall Knight (高大侠) to have some fresh seafood grilled on charcoal.

We ate a bunch of grilled shrimps as appetizers, very large scallops cooked in their shells and oysters the size of a toy football. I'm used to the small and delicious oysters we eat raw in Canada, but this one was served cooked with homemade hot sauce. They also had Hardcore beer (a local Taiwan craft beer) on tap and it went very well with the meal.


Louis-Philippe Véronneau Louis-Philippe Véronneau

Reproducible Builds: Weekly report #168

Planet Debian - Mar, 17/07/2018 - 1:51md

Here’s what happened in the Reproducible Builds effort between Sunday July 8 and Saturday July 14 2018:

Packages reviewed and fixed, and bugs filed diffoscope development

diffoscope is our in-depth “diff-on-steroids” utility which helps us diagnose reproducibility issues in packages. This week, diffoscope version 99 was uploaded to Debian unstable by Mattia Rizzolo. It includes contributions already covered in previous weeks as well as new ones from:

reprotest development

reprotest is our “end-user” tool to build arbitrary software and check it for reproducibility. This week, version 0.7.8 was uploaded to Debian unstable by Mattia Rizzolo. It includes contributions already covered in previous weeks as additional contributions from Mattia, including:


This week’s edition was written by Bernhard M. Wiedemann, Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Reproducible builds folks

pinp 0.0.6: Two new options

Planet Debian - Mar, 17/07/2018 - 12:22md

A small feature release of our pinp package for snazzier one or two column vignettes get onto CRAN a little earlier.

It offers two new options. Saghir Bashir addressed a longer-standing help needed! issue and contributed code to select papersize options via the YAML header. And I added support for the collapse option of knitr, also via YAML header selection.

A screenshot of the package vignette can be seen below. Additional screenshots of are at the pinp page.

The NEWS entry for this release follows.

Changes in pinp version 0.0.6 (2018-07-16)
  • Added YAML header option 'papersize' (Saghir Bashir in #54 and #58 fixing #24).

  • Added YAML header option 'collapse' with default 'false' (#59).

Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel Thinking inside the box

PGP Clean Room Beta

Planet Debian - Mar, 17/07/2018 - 3:10pd

My Google Summer of Code 2018 Project is finally far enough along that I feel that it should have a proper public beta.

PGP Clean Room

This summer I’m working on the PGP Clean Room Live CD project. The goal of this project is to make it easy to create and maintain an offline GPG key. It creates and backs up your GPG key to USB drives which can be stored in a safe place, and exports subkeys for you to use, either via an export USB or a PGP smartcard. It also allows you to sign other people’s keys, revoke your own keys, and change your keys expiration dates. The live system is built on live-build with an added python application (using GPGME to manage keys) and all networking functionality removed.

Testers Wanted

I would love to get some feedback on this build and how it could be improved. You can report bugs via salsa.d.o


Prebuilt ISOs are available here (sig) or on Google Drive (sig)

The source code is available on salsa.d.o


Main Menu

Load Menu

Advanced Menu

Signing subkey

Picking subkey algorithm

Picking a backup disk (in QEMU)

Jacob Adams Tookmund

A very nice side-project that has come to fruition: Fresh from the 1960s, my father's travel memories

Planet Debian - Mar, 17/07/2018 - 2:03pd

So... Everybody I've interacted with along the last couple of weeks knows I'm basically just too busy. If I'm not tied up with stuff regarding my privacy/anonymity project at the university, I am trying to get the DebConf scheduling, or trying to catch up with my perpetual enemy, mail backlog. Of course, there's also my dayjob — Yes, it's vacation time, but I'm a sysadmin, and it's not like I want to give software updates much of a vacation! Of course, my family goes to Argentina for a couple of weeks while I go to DebConf, so there's quite a bit of work in that sphere as well, and... And... And... Meh, many other things better left unaccounted for ☺
But there's one big extra I was working on, somewhat secretly, over the last two months. I didn't want to openly spill the beans on it until it was delivered in hand to its recipient.
Which happened this last weekend. So, here it is!

During the late 1960s, my father studied his PhD in Israel and had a posdoctoral stay in Sweden. During that time, he traveled through the world during his vacations as much as he could — This book collects his travels through Ethiopia (including what today is Eritrea), Kenya, Uganda, Rwanda, Burundi, Turkey, Afghanistan, Pakistan, Czechoslovakia, Sweden, Norway, Iceland and India. As he took those trips, he wrote chronicles about them, and sent them to Mexico's then-most-important newspaper (Excélsior), which published each of them in four to six parts (except for the Czechoslovakia one, which is a single page, devoted to understanding Prague two years after the Soviet repression and occupation).

I did this work starting from the yellow-to-brown and quite brittle copies of the newspaper he kept stored in a set of folders. I had the help of a digitalization professional that often works for the University, but still did a couple of cleanup and QA reads (and still, found typos... In the first printed page, in the first title! :-/ ). The text? Amazing. I thoroughly enjoyed it. He wrote the chronicles being between 23 and 27 years old, but the text flows quick and easy, delightful, as if coming from a professional writer. If you can read Spanish, I am sure you will enjoy the read:

Chronicles of a backpacker in a more naïve world

Why am I publishing this now, amid the work craze I've run into? Because my father is turning 75 year old next weekend. We rushed the mini-party for him (including the book-as-a-present) as we wanted my kids to deliver the present, and they are now in a plane to South America.

The book run I did was quite limited — Just 30 items, to give away to family and close friends. I can, of course, print more on demand. But I want to take this work to a publisher — There are many reasons I believe these youth chronicles are of general interest.

AttachmentSize libro_bwolf.jpg64.11 KB portada.jpg368.39 KB cronicas_de_un_mochilero.pdf25.3 MB gwolf Gunnar Wolf

Review: Effective Python

Planet Debian - Hën, 16/07/2018 - 3:25pd

Review: Effective Python, by Brett Slatkin

Publisher: Addison-Wesley Copyright: 2015 ISBN: 0-13-403428-7 Format: Trade paperback Pages: 216

I'm still looking for a programming language book that's as good as Joshua Bloch's Effective Java, which goes beyond its surface mission to provide valuable and deep advice about how to think about software construction and interface design. Effective Python is, sadly, not that book. It settles for being a more pedestrian guide to useful or tricky corners of Python, with a bit of style guide attached (although not as much as I wanted).

Usually I read books like this as part of learning a language, but in this case I'd done some early experimenting with Python and have been using it extensively for my job for about the past four years. I was therefore already familiar with the basics and with some coding style rules, which made this book less useful. This is more of an intermediate than a beginner's book, but if you're familiar with list and hash comprehensions, closures, standard method decorators, context managers, and the global interpreter lock (about my level of experience when I started reading), at least half of this book will be obvious and familiar material.

The most useful part of the book for me was a deep look at Python's object system, including some fully-worked examples of mix-ins, metaclasses, and descriptors. This material was new to me and a bit different than the approach to similar problems in other programming languages I know. I think this is one of the most complex and hard-to-understand parts of Python and will probably use this as a reference the next time I have to deal with complex class machinery. (That said, this is also the part of Python that I think is the hardest to read and understand, so most programs are better off avoiding it.) The description of generators and coroutines was also excellent, and although the basic concepts will be familiar to most people who have done parallelism in other languages, Slatkin's treatment of parallelism and its (severe) limitations in Python was valuable.

But there were also a lot of things that I was surprised weren't covered. Some of these are due to the author deciding to limit the scope to the standard library, so testing only covers unittest and not the (IMO far more useful) pytest third-party module. Some are gaps in the language that the author can't fix (Python's documentation situation for user-written modules is sad). But there was essentially nothing here about distutils or how to publish modules properly, almost nothing about good namespace design and when to put code into (a topic crying out for some opinionated recommendations), and an odd lack of mention of any static analysis or linting tools. Most books of this type I've read are noticeably more comprehensive and have a larger focus on how to share your code with others.

Slatkin doesn't even offer much of a style guide, which is usually standard in a book of this sort. He does steer the reader away from a few features (such as else with for loops) and preaches the merits of decomposition and small functions, among other useful tidbits. But it falls well short of Damian Conway's excellent guide for Perl, Perl Best Practices.

Anyone who already knows Python will be wondering how Slatkin handles the conflict between Python 2 and Python 3. The answer is that it mostly doesn't matter, since Slatkin spends little time on the parts of the language that differ. In the few places it matters, Effective Python discusses Python 3 first and then mentions the differences or gaps in Python 2. But there's no general discussion about differences between Python 2 and 3, nor is there any guide to updating your own programs or attempting to be compatible with both versions. That's one of the more common real-world problems in Python at the moment, and was even more so when this book was originally written, so it's an odd omission.

Addison-Wesley did a good job on the printing, including a nice, subtle use of color that made the physical book enjoyable to read. But the downside is that this book has a surprisingly expensive retail ($40 USD) for a fairly thin trade paperback. At the time of this writing, Amazon has it on sale at 64% off, which takes the cost down to about the right territory for what you get.

I'm not sorry I read this, and I learned a few things from it despite having used Python fairly steadily for the last few years. But it's nowhere near good enough to recommend to every Python programmer, and with a bit of willingness to search out on-line articles and study high-quality code bases, you can skip this book entirely and never miss it. I found it oddly unopinionated and unsatisfying in the places where I wish Python had more structure or stronger conventions. This is particularly strange given that it was written by a Google staff engineer and Google has a quite comprehensive and far more opinionated coding style guide for Python.

If you want to dig into some of Python's class and object features or see a detailed example of how to effectively use coroutines, Effective Python is a useful guide. Otherwise, you'll probably learn some things from this book, but it's not going to significantly change how you approach the language.

Rating: 6 out of 10

Russ Allbery Eagle's Path

RcppClassic 0.9.11

Planet Debian - Hën, 16/07/2018 - 1:44pd

A new maintenance release, now at version 0.9.11, of the RcppClassic package arrived earlier today on CRAN. This package provides a maintained version of the otherwise deprecated initial Rcpp API which no new projects should use as the normal Rcpp API is so much better.

Per another request from CRAN, we updated the source code in four places to no longer use dynamic exceptions specification. This is something C++11 deprecated, and g++-7 and above now complain about each use. No other changes were made.

CRANberries also reports the changes relative to the previous release.

Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Dirk Eddelbuettel Thinking inside the box

Announcing git-cinnabar 0.5.0 beta 4

Planet Debian - Hën, 16/07/2018 - 1:04pd

Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git.

Get it on github.

These release notes are also available on the git-cinnabar wiki.

What’s new since 0.5.0 beta 3?
  • Fixed incompatibility with Mercurial 3.4.
  • Performance and memory consumption improvements.
  • Work around networking issues while downloading clone bundles from Mozilla CDN with range requests to continue past failure.
  • Miscellaneous metadata format changes.
  • The prebuilt helper for Linux now works across more distributions (as long as is present, it should work)
  • Updated git to 2.18.0 for the helper.
  • Properly support the pack.packsizelimit setting.
  • Experimental support for initial clone from a git repository containing git-cinnabar metadata.
  • Changed the default make rule to only build the helper.
  • Now can successfully clone the pypy and GNU octave mercurial repositories.
  • More user-friendly errors.
glandium p.d.o –

Taiwan Travel Blog - Day 2 & 3

Planet Debian - Mar, 10/07/2018 - 6:00pd

My Taiwan Travel blog continues! I was expecting the weather to go bad on July 10th, but the typhoon arrived late and the rain only started around 20:00. I'm pretty happy because that means I got to enjoy another beautiful day of hiking in Taroko National Park.

I couldn't find time on the 10th to sit down and blog about my trip, so this blog will also include what I did on the 11th.

Xiaozhuilu Trail (小锥麓步道)

The first path I did on the 10th was Xiaozhuilu to warm my muscles a little bit. It links the Shakadang Trail to the Taroko Visitor center and it's both easy and enjoyable. The path is mainly composed of stairs and man-made walkways, but it's the middle of the forest and goes by the LiWu river.

To me, the highlight of the trail was the short rope suspension bridge. How cute!

Dekalun Trail (得卡伦步道)

Once I finished the Xiaozhuilu trail, I decided I was ready for something a little more challenging. Since the park was slowly closing down because of the incoming Typhoon Maria, the only paths I could do were the ones where I didn't need to ride a bus.

I thus started climbing the Dekalun Trail, situated right behind the Taroko Visitor Center.

Although the path is very steep and goes through the wild forest/jungle, this path is also mainly man-made walkways and stairs. Here is a forest interpretation poster I really liked:

The leaves of a tree are its name cards. The name cards of the Macaranga tree are very special. They are large and round and the petiole is not on the leaf margin, it is inside the leaf blade. They are called perforated leaves and look like shields. [...] The Macaranga tree is like a spearhead. When the village here relocated and the fields were abandoned, it quickly moved in. The numerous leaves form a large umbrella that catches a large amount of sunlight and allows it to grow quickly. It can be predicted that in the future, the Macaranga will gradually be replaced by trees that are more shade tolerant. In the meantime however, its leaves, flowers and fruits are a source of food loved by the insects and birds.

A very fengshui tree yo.

Here is a bonus video of one of the giant spiders I was describing yesterday being eaten by ants. For size comparison, the half step you can see is about 10cm large...

Video of a huge dead spider being eaten by ants. Dali - Datong Trail (大礼-大同步道)

The Dekalun Trail ends quite abruptly and diverges into two other paths: one that goes back down and the other one that climbs to the Dali village and then continues to the Datong village.

It was still early in the afternoon when I arrived at the crossroad so I decided that I was at least going to make it to Dali before turning back. Turns out that was a good idea, since the Dali path was a really beautiful mountainside path with a very challenging heigh difference. If the Dekalun Trail is a light 3/5, I'd say the Dali trail is a heavy 3/5. Although I'm in shape, I had to stop multiple times to sit down and try to cool myself. By itself the trail would be fine, but it's the 35+°C with a high level of humidity that made it challenging to me.

Once I arrived at Dali, I needed a permit to continue to Datong but the path was very easy, the weather beautiful and the view incredible, so I couldn't stop myself. I think I walked about half of the 6km trail from Dali to Datong before running out of water. Turns out 4L wasn't enough. The mixed guilt of not having a mountain permit and the concern I wouldn't have anything left to drink for a while made me turn back and start climbing down.

Still, no regrets! This trail was clearly the best one I did so far.

A Wild Andrew Appears!

So there I was in my bed after a day of hiking in the mountains, ready to go to sleep when Andrew Lee reached out to me.

He decided to come by my hostel to talk about the DebConf18 daytrip options. Turns out I'll be the one to lead the River Tracing daytrip on the MeiHua river (梅花溪). River tracing is a mix of bouldering and hiking, but in a river bed.

I'm a little apprehensive of taking the lead of the daytrip since I don't know if my mandarin will be good enough to fully understand the bus driver and the activity guide, but I'll try my best!

Anyway, once we finished talking about the daytrip, Andrew proposed we go to the Hualien night market. After telling him I wasn't able to rent a bike because of the incoming typhoon (nobody would rent me one), we swerved by Carrefour (a large super market chain) and ended up buying a bicycle! The clerk was super nice and even gave me a lock to go with it.

I'm now the proud owner of a brand new Giant bicycle for the rest of my trip in Taiwan. I'm retrospective, I think this was a pretty good idea. It'll end up cheaper than renting one for a large amount of time and will be pretty useful to get around during DebConf.1 It's a little small for me, but I will try to buy a longer seat post in Hualien.

Music and Smoked Flying Fish

After buying the bike, I guess we said fuck the night market and met up with one of Andrew's friend who is a musician and owns a small recording studio. We played music with him for a while and sang songs, and then went back to Andrew's place to eat some flying fish that Andrew had smoked. We drank a little and I decided to sleep there because it was getting pretty late.

Andrew was a wonderful guest and brought me back to my hostel the next day in the afternoon after showing me the Hualien beach and drinking some tea in a local teashop with me. I had a very good time.

What an eventful two days that was! Turns out the big typhoon that was supposed to hit on the 11th turned out to be a fluke and passed to the north of Taiwan: in Hualien we only had a little bit of rain. So much for the rainpocalyspe I was expecting!

Language Rant bis

Short but heartfelt language rant: Jesus Christ on a paddle-board, communication in a language you don't really master is exhausting. I recently understood one of the sentences I was trying to decipher was a pun and I laughed. Then cried a little.

  1. If you plan to stay in Taiwan after DebConf and need a bicycle, I would be happy to sell it for 1500 NTD$ (40€), half of what I paid. It's a little bit cheap, but it's brand new and comes with a 1 year warranty! Better than walking if you ask me. 

Louis-Philippe Véronneau Louis-Philippe Véronneau

Still not going to Debconf....

Planet Debian - Hën, 09/07/2018 - 10:35md

I was looking forward to this year's Debconf in Taiwan, the first in Asia, and the perspective of attending it with no jet lag, but I happen to be moving to Okinawa and changing jobs on August 1st, right at the middle of it...

Moving is a mixed feeling of happiness and excitation for what I am about to find, and melancholy about what and whom I am about to leave. But flights to Tôkyô and Yokohama are very affordable.

Special thanks to the Tôkyô Debian study group, where I got my GPG key signed by Debian developers a long time ago

Charles Plessy Planet

Run Ubuntu on Windows, even multiple releases in parallel!

Planet Debian - Hën, 09/07/2018 - 9:50md

Running Linux terminals on Windows needs just a few clicks since we can install Ubuntu, Debian and other distributions right from the Store as apps, without the old days’ hassle of dual-booting or starting virtual machines. It just works and it works even in enterprise environments where installation policies are tightly controlled.

If you check the Linux distribution apps based on the Windows Subsystem for Linux technology you may notice that there is not only one Ubuntu app, but there are already three, Ubuntu, Ubuntu 16.04 and Ubuntu 18.04. This is no accident. It matches the traditional Ubuntu release offering where the LTS releases are supported for long periods and there is always a recommended LTS release for production:

  • Ubuntu 16.04 (code name: Xenial) was the first release really rocking on WSL and it will be updated in the Store until 16.04’s EOL, April, 2021.
  • Ubuntu 18.04 (code name: Bionic) is the current LTS release (also rocking :-)) and the first one supporting even ARM64 systems on Windows. It will be updated in the Store until 18.04’s EOL, April, 2023.
  • Ubuntu (without the release version) always follows the recommended release, switching over to the next one when it gets the first point release. Right now it installs Ubuntu 16.04 and will switch to 18.04.1, on 26th July, 2018.

The apps in the Store are like installation kits. Each app creates a separate root file system in which Ubuntu terminals are opened but app updates don’t change the root file system afterwards. Installing a different app in parallel creates a different root file system allowing you to have both Ubuntu LTS releases installed and running in case you need it for keeping compatibility with other external systems. You can also upgrade your Ubuntu 16.04 to 18.04 by running ‘do-release-upgrade’ and have three different systems running in parallel, separating production and sandboxes for experiments.

What amazes me in the WSL technology is not only that Linux programs running directly on Windows perform surprisingly well (benchmarks), but the coverage of programs you can run unmodified without any issues and without the large memory overhead of virtual machines.

I hope you will enjoy the power or the Linux terminals on Windows at least as much we enjoyed building the apps at Canonical working closely with Microsoft to make it awesome!

Réczey Bálint debian – Obsessed with reality

My Free Software Activities in June 2018

Planet Debian - Hën, 09/07/2018 - 8:06md

Welcome to Here is my monthly report that covers what I have been doing for Debian. If you’re interested in Java, Games and LTS topics, this might be interesting for you.

Debian Games
  • I advocated Phil Morrell to become Debian Maintainer with whom I have previously worked together on corsix-th. This month I sponsored his updates for scorched3d and the new package, an installer for drm-free commercial games. is basically a collection of shell scripts that create a wrapper around games from or Steam and put them into a Debian package which is then seamlessly integrated into the user’s system.  Similar software are game-data-packager, playonlinux or lutris (not yet in Debian).
  • I packaged new upstream releases of blockattack, renpy, atomix and minetest, and also backported Minetest version to Stretch later on.
  • I uploaded RC bug fixes from Peter de Wachter for torus-trooper, tumiki-fighters and val-and-rick and moved the packages to Git.
  • I tackled an RC bug (#897548) in yabause, a Saturn emulator.
  • I sponsored connectagram, cutemaze and tanglet updates for Innocent de Marchi.
  • Last but not least I refreshed the packaging of trophy and sauerbraten which had not seen any updates for the last couple of years.
Debian Java
  • I packaged a new upstream release of activemq and could later address #901366 thanks to a bug report by Chris Donoghue.
  • I also packaged upstream releases of bouncycastle, libpdfbox-java, libpdfbox2-java because of reported security vulnerabilities.
  • I investigated and fixed RC bugs in openjpa (#901045), osgi-foundation-ee (#893382) and ditaa (#897494, Java 10 related).
  • A snakeyaml update introduced a regression in apktool (#902666) which was only visible at runtime. Once known I could fix it.
  •   I worked on Netbeans again. It can be built from source now but there is still a runtime error (#891957) that prevents users from starting the application. The current plan is to package the latest release candidate of Netbeans 9 and move forward.
Debian LTS

This was my twenty-eight month as a paid contributor and I have been paid to work 23,75 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • From 18.06.2018 until 24.06.2018 I was in charge of our LTS frontdesk. I investigated and triaged CVE in jasperreports, 389-ds-base, asterisk, lava-server, libidn, php-horde-image, tomcat8, thunderbird, glusterfs, ansible, mercurial, php5, jquery, redis, redmine, libspring-java, php-horde-crypt, mupdf, binutils, jetty9 and libpdfbox-java.
  • DSA-4221-1. Issued a security update for libvncserver fixing 1 CVE.
  • DLA-1398-1. Issued a security update for php-horde-crypt fixing 2 CVE.
  • DLA-1399-1. Issued a security update for ruby-passenger fixing 2 CVE.
  • DLA-1411-1. Issued a security update for tiff fixing 5 CVE.
  • DLA-1410-1. Issued a security update for python-pysaml fixing 2 CVE.
  • DLA-1418-1. Issued a security update for bouncycastle fixing 7 CVE.

Extended Long Term Support (ELTS) is a new project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 7 „Wheezy“. This was my first month and I have been paid to work 7 hours on ELTS.

  • ELA-1-1. Issued a security update for Git fixing 1 CVE.
  • ELA-8-1. Issued a security update for ruby-passenger fixing 1 CVE.
  • ELA-14-1. Backported the Linux 3.16 kernel from Jessie to Wheezy. This update also included backports of initramfs-tools and the linux-latest source package. The new kernel is available for amd64 and i386 architectures.
  • I prepared security updates for libvncserver (Stretch, DSA-4221-1) and Sid) and bouncycastle (Stretch, DSA-4233-1)

Thanks for reading and see you next time.

Apo planetdebian –

What is the most supported MIME type in Debian in 2018?

Planet Debian - Hën, 09/07/2018 - 8:05pd

Five years ago, I measured what the most supported MIME type in Debian was, by analysing the desktop files in all packages in the archive. Since then, the DEP-11 AppStream system has been put into production, making the task a lot easier. This made me want to repeat the measurement, to see how much things changed. Here are the new numbers, for unstable only this time:

Debian Unstable:

count MIME type ----- ----------------------- 56 image/jpeg 55 image/png 49 image/tiff 48 image/gif 39 image/bmp 38 text/plain 37 audio/mpeg 34 application/ogg 33 audio/x-flac 32 audio/x-mp3 30 audio/x-wav 30 audio/x-vorbis+ogg 29 image/x-portable-pixmap 27 inode/directory 27 image/x-portable-bitmap 27 audio/x-mpeg 26 application/x-ogg 25 audio/x-mpegurl 25 audio/ogg 24 text/html

The list was created like this using a sid chroot: "cat /var/lib/apt/lists/*sid*_dep11_Components-amd64.yml.gz| zcat | awk '/^ - \S+\/\S+$/ {print $2 }' | sort | uniq -c | sort -nr | head -20"

It is interesting to see how image formats have passed text/plain as the most announced supported MIME type. These days, thanks to the AppStream system, if you run into a file format you do not know, and want to figure out which packages support the format, you can find the MIME type of the file using "file --mime <filename>", and then look up all packages announcing support for this format in their AppStream metadata (XML or .desktop file) using "appstreamcli what-provides mimetype <mime-type>. For example if you, like me, want to know which packages support inode/directory, you can get a list like this:

% appstreamcli what-provides mimetype inode/directory | grep Package: | sort Package: anjuta Package: audacious Package: baobab Package: cervisia Package: chirp Package: dolphin Package: doublecmd-common Package: easytag Package: enlightenment Package: ephoto Package: filelight Package: gwenview Package: k4dirstat Package: kaffeine Package: kdesvn Package: kid3 Package: kid3-qt Package: nautilus Package: nemo Package: pcmanfm Package: pcmanfm-qt Package: qweborf Package: ranger Package: sirikali Package: spacefm Package: spacefm Package: vifm %

Using the same method, I can quickly discover that the Sketchup file format is not yet supported by any package in Debian:

% appstreamcli what-provides mimetype application/vnd.sketchup.skp Could not find component providing 'mimetype::application/vnd.sketchup.skp'. %

Yesterday I used it to figure out which packages support the STL 3D format:

% appstreamcli what-provides mimetype application/sla|grep Package Package: cura Package: meshlab Package: printrun %

PS: A new version of Cura was uploaded to Debian yesterday.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Petter Reinholdtsen Petter Reinholdtsen - Entries tagged english

Taiwan Travel Blog - Day 1

Planet Debian - Hën, 09/07/2018 - 6:00pd

I'm going to DebConf18 later this month, and since I had some free time and I speak a somewhat understandable mandarin, I decided to take a full month of vacation in Taiwan.

I'm not sure if I'll keep blogging about this trip, but so far it's been very interesting and I felt the urge to share the beauty I've seen with the world.

This was the first proper day I spent in Taiwan. I arrived on the 8th during the afternoon, but the time I had left was all spent traveling to Hualien County (花蓮縣) were I intent to spend the rest of my time before DebConf.

Language Rant

I'm mildly annoyed at Taiwan for using traditional Chinese characters instead of simplified ones like they do in Mainland China. So yeah, even though I've been studying mandarin for a while now, I can't read much if anything at all. For those of you not familiar with mandarin, here is an example of a very common character written with simplified (后) and traditional characters (後). You don't see the resemblance between the two? Me neither.

I must say technology is making my trip much easier though. I remember a time when I had to use my pocket dictionary to lookup words and characters and it used to take me up to 5 minutes to find a single character1. That's how you end up ordering cold duck blood soup from a menu without pictures after having given up on translating it.

Now, I can simply use my smartphone and draw the character I'm looking for in my dictionary app. It's fast, it's accurate and it's much more complete than a small pocket dictionary.

Takoro National Park (太鲁阁国家公园)

Since I've seen a bunch of large cities in China already and I both dislike pollution and large amounts of people squished up in too few square meters, I rapidly decided I wasn't going to visit Taipei and would try to move out and explore one of the many national parks in Taiwan.

After looking it up, Takoro National Park in the Hualien County seemed the best option for an extended stay. It's large enough that there is a substantial tourism economy built around visiting the multiple trails of the park, there are both beginner and advanced trails you can choose from and the scenery is incredible.

Also Andrew Lee lives nearby and had a bunch of very nice advice for me, making my trip to Takoro much easier.

Swallow Gorge (燕子口)

The first trail I visited in the morning was Swallow Gorge. Apparently it's frequently closed because of falling rocks. Since the weather was very nice and the trail was open, I decided to start by this one.

Fun fact, at first I thought the swallow in Swallow Gorge meant swallowing, but it is swallow as in the cute bird commonly associated with spring time. The gorge is named that way because the small holes in the cliffs are used by swallows to nest. I kinda understood that when I saw a bunch of them diving and playing in the wind in front of me.

The Gorge was very pretty, but it was full of tourists and the "trail" was actually a painted line next to the road where car drives. It was also pretty short. I guess that's ok for a lot of people, but I was looking for something a little more challenging and less noisy.

Shakadang Trail (砂卡礑步道)

The second trail I visited was the Shakadang trail. The trail dates back to 1940, when the Japanese tried to use the Shakadang river for hydroelectricity.

This trail was very different from Yanzikou, being in the wild and away from cars. It was a pretty easy trail (2/5) and although part of it was paved with concrete, the more you went the wilder it got. In fact, most of the tourist gave up after the first kilometer and I had the rest of the path to myself afterwards.

The path is home to a variety of wild animals, plants and insects. I didn't see any wild board, but gosh damn did I saw some freakingly huge spiders. As I learnt later, Taiwan is home of the largest spiders in the world. The ones I saw (Golden silk orb-weaver, Nephila pilipes) had bodies easily 3 to 5cm long and 2cm thick, with an overall span of 20cm with their legs.

I also heard some bugs (I guess it was bugs) making a huge racket that somewhat reminded me of an old car's loose alternator belt strap on a cold winter morning.

  1. Using a Chinese dictionary is a hard thing to do since there is no alphabet. Instead, the characters are classified by the number of strokes in their radicals and then by the number of strokes in the rest of the character. 

Louis-Philippe Véronneau Louis-Philippe Véronneau

uwsgi; oh my!

Planet Debian - Hën, 09/07/2018 - 5:45pd

The world of Python based web applications, WSGI, its interaction with uwsgi and various deployment methods can quickly turn into a incredible array of confusingly named acronym soup. If you jump straight into the uwsgi documentation it is almost certain you will get lost before you start!

Below tries to lay out a primer for the foundations of application deployment within devstack; a tool for creating a self-contained OpenStack environment for testing and interactive development. However, it is hopefully of more general interest for those new to some of these concepts too.


Let's start with WSGI. Fully described in PEP 333 -- Python Web Server Gateway Interface the core concept a standardised way for a Python program to be called in response to a web request. In essence, it bundles the parameters from the incoming request into known objects, and gives you can object to put data into that will get back to the requesting client. The "simplest application", taken from the PEP directly below, highlights this perfectly:

def simple_app(environ, start_response): """Simplest possible application object""" status = '200 OK' response_headers = [('Content-type', 'text/plain')] start_response(status, response_headers) return ['Hello world!\n']

You can start building frameworks on top of this, but yet maintain broad interoperability as you build your application. There is plenty more to it, but that's all you need to follow for now.

Using WSGI

Your WSGI based application needs to get a request from somewhere. We'll refer to the diagram below for discussions of how WSGI based applications can be deployed.

In general, this is illustrating how an API end-point might be connected together to an underlying WSGI implementation written in Python ( Of course, there are going to be layers and frameworks and libraries and heavens knows what else in any real deployment. We're just concentrating on Apache integration -- the client request hits Apache first and then gets handled as described below.


Starting with 1 in the diagram above, we see CGI or "Common Gateway Interface". This is the oldest and most generic method of a web server calling an external application in response to an incoming request. The details of the request are put into environment variables and whatever process is configured to respond to that URL is fork() -ed. In essence, whatever comes back from stdout is sent back to the client and then the process is killed. The next request comes in and it starts all over again.

This can certainly be done with WSGI; above we illustrate that you'd have a framework layer that would translate the environment variables into the python environ object and connect up the processes output to gather the response.

The advantage of CGI is that it is the lowest common denominator of "call this when a request comes in". It works with anything you can exec, from shell scripts to compiled binaries. However, forking processes is expensive, and parsing the environment variables involves a lot of fiddly string processing. These become issues as you scale.


Illustrated by 2 above, it is possible to embed a Python interpreter directly into the web server and call the application from there. This is broadly how mod_python, mod_wsgi and mod_uwsgi all work.

The overheads of marshaling arguments into strings via environment variables, then unmarshaling them back to Python objects can be removed in this model. The web server handles the tricky parts of communicating with the remote client, and the module "just" needs to translate the internal structures of the request and response into the Python WSGI representation. The web server can manage the response handlers directly leading to further opportunities for performance optimisations (more persistent state, etc.).

The problem with this model is that your web server becomes part of your application. This may sound a bit silly -- of course if the web server doesn't take client requests nothing works. However, there are several situations where (as usual in computer science) a layer of abstraction can be of benefit. Being part of the web server means you have to write to its APIs and, in general, its view of the world. For example, mod_uwsgi documentation says

"This is the original module. It is solid, but incredibly ugly and does not follow a lot of apache coding convention style".


mod_python is deprecated with mod_wsgi as the replacement. These are obviously tied very closely to internal Apache concepts.

In production environments, you need things like load-balancing, high-availability and caching that all need to integrate into this model. Thus you will have to additionally ensure these various layers all integrate directly with your web server.

Since your application is the web server, any time you make small changes you essentially need to manage the whole web server; often with a complete restart. Devstack is a great example of this; where you have 5-6 different WSGI-based services running to simulate your OpenStack environment (compute service, network service, image service, block storage, etc) but you are only working on one component which you wish to iterate quickly on. Stopping everything to update one component can be tricky in both production and development.


Which brings us to uwsgi (I call this "micro-wsgi" but I don't know if it actually intended to be a μ). uwsgi is a real Swiss Army knife, and can be used in contexts that don't have to do with Python or WSGI -- which I believe is why you can get quite confused if you just start looking at it in isolation.

uwsgi lets us combine some of the advantages of being part of the web server with the advantages of abstraction. uwsgi is a complete pluggable network daemon framework, but we'll just discuss it in one context illustrated by 3.

In this model, the WSGI application runs separately to the webserver within the embedded python interpreter provided by the uwsgi daemon. uwsgi is, in parts, a web-server -- as illustrated it can talk HTTP directly if you want it to, which can be exposed directly or via a traditional proxy.

By using the proxy extension mod_proxy_uwsgi we can have the advantage of being "inside" Apache and forwarding the requests via a lightweight binary channel to the application back end. In this model, uwsgi provides a uwsgi:// service using its internal protcol on a private port. The proxy module marshals the request into small packets and forwards it to the given port. uswgi takes the incoming request, quickly unmarshals it and feeds it into the WSGI application running inside. Data is sent back via similarly fast channels as the response (note you can equally use file based Unix sockets for local only communication).

Now your application has a level of abstraction to your front end. At one extreme, you could swap out Apache for some other web server completely and feed in requests just the same. Or you can have Apache start to load-balance out requests to different backend handlers transparently.

The model works very well for multiple applications living in the same name-space. For example, in the Devstack context, it's easy with mod_proxy to have Apache doing URL matching and separate out each incoming request to its appropriate back end service; e.g.

  • http://service/identity gets routed to Keystone running at localhost:40000
  • http://service/compute gets sent to Nova at localhost:40001
  • http://service/image gets sent to glance at localhost:40002

and so on (you can see how this is exactly configured in lib/apache:write_uwsgi_config).

When a developer makes a change they simply need to restart one particular uwsgi instance with their change and the unified front-end remains untouched. In Devstack (as illustrated) the uwsgi processes are further wrapped into systemd services which facilitates easy life-cycle and log management. Of course you can imagine you start getting containers involved, then container orchestrators, then clouds-on-clouds ...


There's no right or wrong way to deploy complex web applications. But using an Apache front end, proxying requests via fast channels to isolated uwsgi processes running individual WSGI-based applications can provide both good performance and implementation flexibility.

Ian Wienand Technovelty

Fixing a broken ESP8266

Planet Debian - Dje, 08/07/2018 - 4:21md

One of the IoT platforms I’ve been playing with is the ESP8266, which is a pretty incredible little chip with dev boards available for under £4. Arduino and Micropython are both great development platforms for them, but the first board I bought (back in 2016) only had a 4Mbit flash chip. As a result I spent some time writing against the Espressif C SDK and trying to fit everything into less than 256KB so that the flash could hold 2 images and allow over the air updates. Annoyingly just as I was getting to the point of success with Richard Burton’s rBoot my device started misbehaving, even when I went back to the default boot loader:

ets Jan 8 2013,rst cause:1, boot mode:(3,6) load 0x40100000, len 816, room 16 tail 0 chksum 0x8d load 0x3ffe8000, len 788, room 8 tail 12 chksum 0xcf ho 0 tail 12 room 4 load 0x3ffe8314, len 288, room 12 tail 4 chksum 0xcf csum 0xcf 2nd boot version : 1.2 SPI Speed : 40MHz SPI Mode : DIO SPI Flash Size : 4Mbit jump to run user1 Fatal exception (0): epc1=0x402015a4, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000 Fatal exception (0): epc1=0x402015a4, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000 Fatal exception (0):

(repeats indefinitely)

Various things suggested this was a bad flash. I tried a clean Micropython install, a restore of the original AT firmware backup I’d taken, and lots of different combinations of my own code/the blinkenlights demo and rBoot/Espressif’s bootloader. I made sure my 3.3v supply had enough oompf (I’d previously been cheating and using the built in FT232RL regulator, which doesn’t have quite enough when the device is fully operational, rather than in UART boot mode, such as doing an OTA flash). No joy. I gave up and moved on to one of the other ESP8266 modules I had, with a greater amount of flash. However I was curious about whether this was simply a case of the flash chip wearing out (various sites claim the cheap ones on some dev boards will die after a fairly small number of programming cycles). So I ordered some 16Mb devices - cheap enough to make it worth trying out, but also giving a useful bump in space.

They arrived this week and I set about removing the old chip and soldering on the new one (Andreas Spiess has a useful video of this, or there’s Pete Scargill’s write up). Powered it all up, ran flash_id to see that it was correctly detected as a 16Mb/2MB device and set about flashing my app onto it. Only to get:

ets Jan 8 2013,rst cause:2, boot mode:(3,3) load 0x40100000, len 612, room 16 tail 4 chksum 0xfd load 0x88380000, len 565951362, room 4 flash read err, ets_unpack_flash_code ets_main.c

Ooops. I had better luck with a complete flash erase ( erase_flash) and then a full program of Micropython using --baud 460800 write_flash --flash_size=detect -fm dio 0 esp8266-20180511-v1.9.4.bin, which at least convinced me I’d managed to solder the new chip on correctly. Further experimention revealed I needed to pass all of the flash parameters to to get rBoot entirely happy, and include esp_init_data_default.bin (FWIW I updated everything to v2.2.1 as part of the process): write_flash --flash_size=16m -fm dio 0x0 rboot.bin 0x2000 rom0.bin \ 0x120000 rom1.bin 0x1fc000 esp_init_data_default_v08.bin

Which gives (at the default 76200 of the bootloader bit):

ets Jan 8 2013,rst cause:1, boot mode:(3,7) load 0x40100000, len 1328, room 16 tail 0 chksum 0x12 load 0x3ffe8000, len 604, room 8 tail 4 chksum 0x34 csum 0x34 rBoot v1.4.2 - Flash Size: 16 Mbit Flash Mode: DIO Flash Speed: 40 MHz Booting rom 0. rf cal sector: 507 freq trace enable 0 rf[112]

Given the cost of the modules it wasn’t really worth my time and energy to actually fix the broken one rather than buying a new one, but it was rewarding to be sure of the root cause. Hopefully this post at least serves to help anyone seeing the same exception messages determine that there’s a good chance their flash has died, and that a replacement may sort the problem.

Jonathan McDowell Noodles' Emptiness

Debian APT upgrade without enough free space on the disk...

Planet Debian - Dje, 08/07/2018 - 12:10md

Quite regularly, I let my Debian Sid/Unstable chroot stay untouch for a while, and when I need to update it there is not enough free space on the disk for apt to do a normal 'apt upgrade'. I normally would resolve the issue by doing 'apt install <somepackages>' to upgrade only some of the packages in one batch, until the amount of packages to download fall below the amount of free space available. Today, I had about 500 packages to upgrade, and after a while I got tired of trying to install chunks of packages manually. I concluded that I did not have the spare hours required to complete the task, and decided to see if I could automate it. I came up with this small script which I call 'apt-in-chunks':

#!/bin/sh # # Upgrade packages when the disk is too full to upgrade every # upgradable package in one lump. Fetching packages to upgrade using # apt, and then installing using dpkg, to avoid changing the package # flag for manual/automatic. set -e ignore() { if [ "$1" ]; then grep -v "$1" else cat fi } for p in $(apt list --upgradable | ignore "$@" |cut -d/ -f1 | grep -v '^Listing...'); do echo "Upgrading $p" apt clean apt install --download-only -y $p for f in /var/cache/apt/archives/*.deb; do if [ -e "$f" ]; then dpkg -i /var/cache/apt/archives/*.deb break fi done done

The script will extract the list of packages to upgrade, try to download the packages needed to upgrade one package, install the downloaded packages using dpkg. The idea is to upgrade packages without changing the APT mark for the package (ie the one recording of the package was manually requested or pulled in as a dependency). To use it, simply run it as root from the command line. If it fail, try 'apt install -f' to clean up the mess and run the script again. This might happen if the new packages conflict with one of the old packages. dpkg is unable to remove, while apt can do this.

It take one option, a package to ignore in the list of packages to upgrade. The option to ignore a package is there to be able to skip the packages that are simply too large to unpack. Today this was 'ghc', but I have run into other large packages causing similar problems earlier (like TeX).

Update 2018-07-08: Thanks to Paul Wise, I am aware of two alternative ways to handle this. The "unattended-upgrades --minimal-upgrade-steps" option will try to calculate upgrade sets for each package to upgrade, and then upgrade them in order, smallest set first. It might be a better option than my above mentioned script. Also, "aptutude upgrade" can upgrade single packages, thus avoiding the need for using "dpkg -i" in the script above.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

Petter Reinholdtsen Petter Reinholdtsen - Entries tagged english

Getting Started with Debian Packaging

Planet Debian - Dje, 08/07/2018 - 6:00pd

One of my tasks in GSoC involved set up of Thunderbird extensions for the user. Some of the more popular add-ons like ‘Lightning’ (calendar organiser) already has a Debian package.

Another important add on is ‘Cardbook’ which is used to manage contacts for the user based on CardDAV and vCard standards. But it doesn’t have a package yet.

My mentor, Daniel motivated me to create a package for it and upload it to It would ease the installation process as it could get installed through apt-get. This blog describes how I learned and created a Debian package for CardBook from scratch.

Since, I was new to packaging, I did extensive research on basics of building a package from the source code and checked if the license was DFSG compatible.

I learned from various Debian wiki guides like ‘Packaging Intro’, ‘Building a Package’ and blogs.

I also studied the amd64 files included in Lightning extension package.

The package I created could be found here.

Debian Package

Creating an empty package

I started by creating a debian directory by using dh_make command

# Empty project folder $ mkdir -p Debian/cardbook # create files $ dh_make\ > --native \ > --single \ > --packagename cardbook_1.0.0 \ > --email

Some important files like control, rules, changelog, copyright are initialized with it.

The list of all the files created:

$ find /debian debian/ debian/rules debian/preinst.ex debian/ debian/manpage.1.ex debian/install debian/source debian/source/format debian/cardbook.debhelper.lo debian/manpage.xml.ex debian/README.Debian debian/postrm.ex debian/prerm.ex debian/copyright debian/changelog debian/manpage.sgml.ex debian/cardbook.default.ex debian/README debian/cardbook.doc-base.EX debian/README.source debian/compat debian/control debian/debhelper-build-stamp debian/menu.ex debian/postinst.ex debian/cardbook.substvars debian/files

I gained an understanding of Dpkg package management program in Debian and its use to install, remove and manage packages.

I build an empty package with dpkg commands. This created an empty package with four files namely .changes, .deb, .dsc, .tar.gz

.dsc file contains the changes made and signature

.deb is the main package file which can be installed

.tar.gz (tarball) contains the source package

The process also created the README and changelog files in /usr/share. They contain the essential notes about the package like description, author and version.

I installed the package and checked the installed package contents. My new package mentions the version, architecture and description!

$ dpkg -L cardbook /usr /usr/share /usr/share/doc /usr/share/doc/cardbook /usr/share/doc/cardbook/README.Debian /usr/share/doc/cardbook/changelog.gz /usr/share/doc/cardbook/copyright

Including CardBook source files

After successfully creating an empty package, I added the actual CardBook add-on files inside the package. The CardBook’s codebase is hosted here on Gitlab. I included all the source files inside another directory and told the build package command which files to include in the package.

I did this by creating a file debian/install using vi editor and listed the directories that should be installed. In this process I spent some time learning to use Linux terminal based text editors like vi. It helped me become familiar with editing, creating new files and shortcuts in vi.

Once, this was done, I updated the package version in the changelog file to document the changes that I have made.

$ dpkg -l | grep cardbook ii cardbook 1.1.0 amd64 Thunderbird add-on for address book

Changelog file after updating Package

After rebuilding it, dependencies and detailed description can be added if necessary. The Debian control file can be edited to add the additional package requirements and dependencies.

Local Debian Repository

Without creating a local repository, CardBook could be installed with:

$ sudo dpkg -i cardbook_1.1.0.deb

To actually test the installation for the package, I decided to build a local Debian repository. Without it, the apt-get command would not locate the package, as it is not in uploaded in Debian packages on net.

For configuring a local Debian repository, I copied my packages (.deb) to Packages.gz file placed in a /tmp location.

Local Debian Repo

To make it work, I learned about the apt configuration and where it looks for files.

I researched for a way to add my file location in apt-config. Finally I could accomplish the task by adding *.list file with package’s path in APT and updating ‘apt-cache’ afterwards.

Hence, the latest CardBook version could be successfully installed by apt-get install cardbook

CardBook Installation through apt-get

Fixing Packaging errors and bugs

My mentor, Daniel helped me a lot during this process and guided me how to proceed further with the package. He told me to use Lintian for fixing common packaging error and then using dput to finally upload the CardBook package.

Lintian is a Debian package checker which finds policy violations and bugs. It is one of the most widely used tool by Debian Maintainers to automate checks for Debian policies before uploading the package.

I have uploaded the second updated version of the package in a separate branch of the repository on Salsa here inside Debian directory.

I installed Lintian from backports and learned to use it on a package to fix errors. I researched on the abbreviations used in its errors and how to show detailed response from lintian commands

$ lintian -i -I --show-overrides cardbook_1.2.0.changes

Initially on running the command on the .changes file, I was surprised to see that a large number of errors, warnings and notes were displayed!

Brief errors after running Lintian on Package

Detailed Lintian errors (1)

Detailed Lintian errors (2) and many more…

I spend some days to fix some errors related to Debian package policy violations. I had to dig into every policy and Debian rules carefully to eradicate a simple error. For this I referred various sections on Debian Policy Manual and Debian Developer’s Reference.

I am still working on making it flawless and hope to upload it on soon!

It would be grateful if people from the Debian community who use Thunderbird could help fix these errors.

Minkush Jain Minkush Jain


Subscribe to AlbLinux agreguesi