Leaving Canonical for Endless new possibilities

After a little over 6 years, I am embarking on a new adventure. Today is my last day at Canonical, it’s bitter sweet saying goodbye precisely because it has been such a joy and an honor to be working here with so many amazing, talented and friendly people. But I am leaving by choice, and for an opportunity that makes me as excited as leaving makes me sad.

Goodbye Canonical

maltaI’ve worked at Canonical longer than I’ve worked at any company, and I can honestly say I’ve grown more here both personally and professionally than I have anywhere else. It launched my career as a Community Manager, learning from the very best in the industry how to grow, nurture, and excite a world full of people who share the same ideals. I owe so many thanks (and beers) to Jono Bacon, David Planella, Daniel Holbach, Jorge Castro, Nicholas Skaggs, Alan Pope, Kyle Nitzsche and now also Martin Wimpress. I also couldn’t have done any of this without the passion and contributions of everybody in the Ubuntu community who came together around what we were doing.

As everybody knows by now, Canonical has been undergoing significant changes in order to set it down the road to where it needs to be as a company. And while these changes aren’t the reason for my leaving, it did force me to think about where I wanted to go with my future, and what changes were needed to get me there. Canonical is still doing important work, I’m confident it’s going to continue making a huge impact on the technology and open source worlds and I wish it nothing but success. But ultimately I decided that where I wanted to be was along a different path.

Of course I have to talk about the Ubuntu community here. As big of an impact as Canonical had on my life, it’s only a portion of the impact that the community has had. From the first time I attended a Florida LoCo Team event, I was hooked. I had participated in open source projects before, but that was when I truly understood what the open source community was about. Everybody I met, online or in person, went out of their way to make me feel welcome, valuable, and appreciated. In fact, it was the community that lead me to work for Canonical in the first place, and it was the community work I did that played a big role in me being qualified for the job. I want to give a special shout out to Daniel Holbach and Jorge Castro, who built me up from a random contributor to a project owner, and to Elizabeth Joseph and Laura Faulty who encouraged me to take on leadership roles in the community. I’ve made so many close and lasting friendships by being a part of this amazing group of people, and that’s something I will value forever. I was a community member for years before I joined Canonical, and I’m not going anywhere now. Expect to see me around on IRC, mailing lists and other community projects for a long time to come.

Hello Endless

EndlessNext week I will be joining the team at Endless as their Community Manager. Endless is an order of magnitude smaller than Canonical, and they have a young community that it still getting off the ground. So even though I’ll have the same role I had before, there will be new and exciting challenges involved. But the passion is there, both in the company and the community, to really explode into something big and impactful. In the coming months I will be working to setup the tools, processes and communication that will be needed to help that community grow and flourish. After meeting with many of the current Endless employees, I know that my job will be made easier by their existing commitment to both their own community and their upstream communities.

What really drew me to Endless was the company’s mission. It’s not just about making a great open source project that is shared with the world, they have a specific focus on social good and improving the lives of people who the current technology isn’t supporting. As one employee succinctly put it to me: the whole world, empowered. Those who know me well will understand why this resonates with me. For years I’ve been involved in open source projects aimed at early childhood education and supporting those in poverty or places without the infrastructure that most modern technology requires. And while Ubuntu covers much of this, it wasn’t the primary focus. Being able to work full time on a project that so closely aligned with my personal mission was an opportunity I couldn’t pass up.

Broader horizons

Over the past several months I’ve been expanding the number of communities I’m involved in. This is going to increase significantly in my new role at Endless, where I will be working more frequently with upstream and side-stream projects on areas of mutual benefit and interest. I’ve already started to work more with KDE, and I look forward to becoming active in GNOME and other open source desktops soon.

I will also continue to grow my independent project, Phoenicia, which has a similar mission to Endless but a different technology and audience. Now that this is no longer competing in the XPRIZE competition, it releases some restrictions that we had to operate under and frees us to investigate new areas of innovation and collaboration. If you’re interested in game development, or making an impact on the lives of children around the world, come and see what we’re doing.

If anybody wants to reach out to me to chat, you can still reach me at mhall119@ubuntu.com and soon at mhall119@endlessm.com, tweet me at @mhall119, connect on LinkedIn, chat on Telegram or circle me on Google+. And if we’re ever at a conference together give me a shout, I’d love to grab a drink and catch up.

Posted in Community, OpenSource, Work | Tagged , , , | 7 Comments

Machine Learning with Snaps

Late last year Amazon introduce a new EC2 image customized for Machine Learning (ML) workloads. To make things easier for data scientists and researchers, Amazon worked on including a selection of ML libraries into these images so they wouldn’t have to go through the process of downloading and installing them (and often times building them) themselves.

But while this saved work for the researchers, it was no small task for Amazon’s engineers. To keep offering the latest version of these libraries they had to repeat this work every time there was a new release , which was quite often for some of them. Worst of all they didn’t have a ready-made way to update those libraries on instances that were already running!

By this time they’d heard about Snaps and the work we’ve been doing with them in the cloud, so they asked if it might be a solution to their problems. Normally we wouldn’t Snap libraries like this, we would encourage applications to bundle them into their own Snap package. But these libraries had an unusual use-case: the applications that needed them weren’t mean to be distributed. Instead the application would exist to analyze a specific data set for a specific person. So as odd as it may sound, the application developer was the end user here, and the library was the end product, which made it fit into the Snap use case.

Screenshot from 2017-03-23 16-43-19To get them started I worked on developing a proof of concept based on MXNet, one of their most used ML libraries. The source code for it is part C++, part Python, and Snapcraft makes working with both together a breeze, even with the extra preparation steps needed by MXNet’s build instructions. My snapcraft.yaml could first compile the core library and then build the Python modules that wrap it, pulling in dependencies from the Ubuntu archives and Pypi as needed.

This was all that was needed to provide a consumable Snap package for MXNet. After installing it you would just need to add the snap’s path to your LD_LIBRARY_PATH and PYTHONPATH environment variables so it would be found, but after that everything Just Worked! For an added convenience I provided a python binary in the snap, wrapped in a script that would set these environment variables automatically, so any external code that needed to use MXNet from the snap could simply be called with /snap/bin/mxnet.python rather than /usr/bin/python (or, rather, just mxnet.python because /snap/bin/ is already in PATH).

I’m now working with upstream MXNet to get them building regular releases of this snap package to make it available to Amazon’s users and anyone else. The Amazon team is also seeking similar snap packages from their other ML libraries. If you are a user or contributor to any of these libraries, and you want to make it easier than ever for people to get the latest and greatest versions of them, let’s get together and make it happen! My MXNet example linked to above should give you a good starting point, and we’re always happy to help you with your snapcraft.yaml in #snapcraft on rocket.ubuntu.com.

If you’re just curious to try it out ourself, you can download my snap and then follow along with the MXNet tutorial, using the above mentioned mxnet.python for your interactive python shell.

Posted in OpenSource, Projects, Upstream, Work | Tagged , , , , , | Comments Off on Machine Learning with Snaps

War on Snaps

Java is a well established language for developing web applications, in no small part because of it’s industry standard framework for building them: Servlets and JSP.  Another important part of this standard is the Web Archive, or WAR, file format, which defines how to provide a web application’s executables and how they should be run in a way that is independent of the application server that will be running  them.

application-server-market-share-2015WAR files make life easier for developers by separate the web application from the web server. Unfortunately this doesn’t actually make it easier to deploy a webapp, it only shifts some of the burden off of the developers and on to the user, who still needs to setup and configure an application server to host it. One popular option is Apache’s Tomcat webapp server, which is both lightweight and packs enough features to support the needs of most webapps.

And here is where Snaps come in. By combining both the application and the server into a single, installable package you get the best of both, and with a little help from Snapcraft you don’t have to do any extra work.

Snapcraft supports a modular build configuration by having multiple “parts“, each of which provides some aspect of your complete runtime environment in a way that is configurable and reusable. This is extended to a feature called “remote parts” which are pre-defined parts you can easily pull into your snap by name. It’s this combination of reusable and remote parts that are going to make snapping up java web applications incredibly easy.

The remote part we are going to use is the “tomcat” part, which will build the Tomcat application server from upstream source and bundle it in your snap ready to go. All that you, as the web developer, need to provide is your .war file. Below is an simple snapcraft.yaml that will bundle Tomcat’s “sample” war file into a self-contained snap package.

name: tomcat-sample
version: '0.1'
summary: Sample webapp using tomcat part
description: |
 This is a basic webapp snap using the remote Tomcat part

grade: stable
confinement: strict

parts:
  my-part:
    plugin: dump
    source: .
    organize:
      sample.war: ./webapps/sample.war
    after: [tomcat]

apps:
  tomcat:
    command: tomcat-launch
    daemon: simple
    plugs: [network-bind]

The important bits are the ones in bold, let’s go through them one at a time starting with the part named “my-part”. This uses the simple “dump” plugin which is just going to copy everything in it’s source (current directory in this case) into the resulting snap. Here we have just the sample.war file, which we are going to move into a “webapps” directory, because that is where the Tomcat part is going to look for war files.

Now for the magic, by specifying that “my-part” should come after the “tomcat” part (using after: [tomcat]) which isn’t defined elsewhere in the snapcraft.yaml, we will trigger Snapcraft to look for a remote part by that same name, which conveniently exists for us to use. This remote part will do two things, first it will download and build the Tomcat source code, and then it will generate a “tomcat-launch” shell script that we’ll use later. These two parts, “my-part” and “tomcat” will be combined in the final snap, with the Tomcat server automatically knowing about and installing the sample.war webapp.

The “apps” section of the snapcraft.yaml defines the application to be run. In this simple example all we need to execute is the “tomcat-launch” script that was created for us. This sets up the Tomcat environment variables and runtime directories so that it can run fully confined within the snap. And by declaring it to be a simple daemon we are additionally telling it to auto-start as soon as it’s installed (and after any reboot) which will be handled by systemd.

Now when you run “snapcraft” on this config, you will end up with the file tomcat-sample_0.1_amd64.snap which contains your web application, the Tomcat application server, and a headless Java JRE to run it all. That way the only thing your users need to do to run your app is to “snap install tomcat-sample” and everything will be up and running at http://localhost:8080/sample/ right away, no need to worry about installing dependencies or configuring services.

Screenshot from 2017-03-21 14-16-59

If you have a webapp that you currently deploy as a .war file, you can snap it yourself in just a few minutes, use the snapcraft.yaml defined above and replace the sample data with your own. To learn more about Snaps and Snapcraft in general you can follow this tutorial as well as learning how to publish your new snap to the store.

Posted in Uncategorized | Tagged , , , , , | Comments Off on War on Snaps

Make your world a better place

For much of the past year I have been working on a game. No, not just a game, I’m been working on change. There are 122 million children in the world today who can’t read or write[1]. They will grow up to join the 775 million adults who can’t. Together that’s almost one billion people who are effectively shut off from the information age. How many of them could make the world a better place, given even half a chance?

I’ve been interested in the intersection of open source and education for underprivileged children for quite some time.  I even build a a Linux distro towards that end. So when Jono Bacon told me about a new XPRIZE contest to build open source software for teaching literacy skills to children in Africa, of course I was interested.  And now, a little more than a year later, I have a game that I firmly believe can deliver that world changing ambition.

device-2016-08-25-224444This is where you come in. Don’t worry, I’m not going to ask you to help build my contest entry, though it is already open source (GPLv3) and on github. But the contest entries only cover English and Kiswahili, which is going to leave a very large part of the illiterate population out. That’s not enough, to change the world it needs to be available to the world. Additional languages won’t be part of the contest entry, but they will be a part of making the world a better place.

I designed Phoenicia from the beginning to be able to support as many languages as possible, with as little additional work as possible. But while it may be capable of using handling multiple languages, I sadly am not. So I’m reaching out to the community to help me bring literacy to millions more children than I can do by myself. Children who speak your language, live in your community, who may be your own neighbors.

You don’t need to be a programmer, in fact there shouldn’t be any programming work needed at all. What I need are early reader words, for each language. From there I can show you how to build a locale pack, record audio help, and add any new artwork needed to support your localization. I’m especially looking to those of you who speak French, Spanish and Portuguese, as those languages will carry Phoenicia into many countries where childhood illiteracy is still a major problem.

Posted in Community, OpenSource, Programming, Projects | Tagged , , , , , , | 3 Comments

Desktop app snap in 300KB

KDE Neon developer Harald Sitter was able to package up the KDE calculator, kcalc, in a snap that weighs in at a mere 320KB! How did he do it?

KCalc and KDE Frameworks snaps

Like most applications in KDE, kcalc depends on several KDE Frameworks (though not all), sets of libraries and services that provide the common functionality and shared UI/UX found in KDE and it’s suite of applications. This means that, while kcalc is itself a small application, it’s dependency chain is not. In the past, any KDE application snap had to include many megabytes of platforms dependencies, even for the smallest app.

Recently I introduced the new “content” interface that has been added to snapd. I used this interface to share plugin code with a text editor, but Harald has taken it even further and created a KDE Frameworks snap that can share the entire platform with applications that are built on it!

While still in the very early stages of development, this approach will allow the KDE project to deliver all of their applications as independent snaps, while still letting them all share the one common set of Frameworks that they depend on. The end result will be that you, the user, will get the very latest stable (or development!) version of the KDE platform and applications, direct from KDE themselves, even if you’re on a stable/LTS release of your distro.

If you are running a snap-capable distro, you can try these experimental packages yourself by downloading kde-frameworks-5_5.26_amd64.snap and kcalc_0_amd64.snap from Neon’s build servers, and installing them with “snap install –devmode –force-dangerous <snap_file>”. To learn more about how he did this, and to help him build more KDE application snaps, you can find Harald as <sitter> on #kde-neon on Freenode IRC.

Posted in Community, Upstream, Work | Tagged , , , , , , , | 7 Comments

Sharing is caring, with Snaps!

Snaps are a great way to get the most up to date applications on your desktop without putting the security or stability or your system at risk. I’ve been snapping up a bunch of things lately and the potential this new paradigm offers is going to be revolutionary. Unfortunately nothing comes for free, and the security of snaps comes with some necessary tradeoffs like isolation and confinement, which reduces some of the power and flexibility we’ve become used to as Linux users.

But now the developers of the snappy system (snapd, snap-confine and snapcraft) are giving us back some of that missing flexibility in the form of a new “content” interface which allows you to share files (executables, libraries, or data) between the snap packages that you develop. I decided to take this new interface for a test drive using one of the applications I had recently snapped: Geany, my editor of choice. Geany has the ability to load plugins to extend it’s functionality, and infact has a set of plugins available in a separate Github repository from the application itself.

I already had a working snap for Geany, so the next thing I had to do was create a snap for the plugins. Like Geany itself, the plugins are hosted on GitHub and have a nice build configuration already, so turning it into a snap was pretty trivial. I used the autotools plugin in Snapcraft to pull the git source and build all of the available plugins. Because my Geany snap was built with Gtk+ 3, I had to build the plugins for the same toolkit, but other than that I didn’t have to do anything special.

parts:
 all-plugins:
 plugin: autotools
 source: git@github.com:geany/geany-plugins.git
 source-type: git
 configflags: [--enable-gtk3=yes --enable-all-plugins]

Now that I had a geany.snap and geany-plugins.snap, the next step was to get them working together. Specifically I wanted Geany to be able to see and load the plugin files from the plugins snap, so it was really just a one-way sharing. To do this I had to create both a slot and a plug using the content interface. Usually when you’re building snap you only use plugs, such as network or x11, because you are consuming services provided by the core OS. In those cases also you just have to provide the interface name in the list of plugs, because the interface and the plug have the same name.

But with the content interface you need to do more than that. Because different snaps will provide different content, and a single snap can provide multiple kinds of content, you have to define a new name that is specific to what content you are sharing. So in my geany-plugins snapcraft.yaml I defined a new kind of content that I called geany-plugins-all (because it contains all the geany plugins in the snap), and I put that into a slot called geany-plugins-slot which is how we will refer to it later. I told snapcraft that this new slot was using the content interface, and then finally told it what content to share across that interface, which for geany-plugins was the entire snap’s content.

slots:
 geany-plugins-slot:
 content: geany-plugins-all
 interface: content
 read:
 - /

With that I had one half of the content interface defined. I had a geany-plugins.snap that was able to share all of it’s content with another snap. The next step was to implement the plug half of the interface in my existing geany.snap. This time instead of using a slots: section I would define a plugs: section, with a new plug named geany-plugins-plug and again specifying the interface to be content just like in the slot. Here again I had to specify the content by name, which had to match the geany-plugins-all that was used in the slot. The names of the plug and slot are only relevant to the user who needs to connect them, it’s this content name that snapd uses to make sure they can be connected in the first place. Finally I had to give the plug a target directory for where the shared content will be put. I chose a directory called plugins, and when the snaps are connected the geany-plugins.snap content will be bind-mounted into this directory in the geany.snap

plugs:
 geany-plugins-plug:
 content: geany-plugins-all
 default-provider: geany-plugins
 interface: content
 target: plugins

Lastly I needed to tell snapcraft which app would use this interface. Since the Geany snap only has one, I added it there.

apps:
 geany:
 command: gtk-launch geany
 plugs: [x11, unity7, home, geany-plugins-plug]

Once the snaps were built, I could install them and the new plug and slot were automatically connected

$ snap interfaces
Slot                             Plug
geany-plugins:geany-plugins-slot geany:geany-plugins-plug

Now that put the plugins into the application’s snap space, but it wasn’t enough for Geany to actually find them. To do that I used Geany’s Extra plugin path preferences to point it to the location of the shared plugin files.

Screenshot from 2016-08-30 16-27-12

After doing that, I could open the Plugin manager and see all of the newly shared plugins. Not all of them work, and some assume specific install locations or access to other parts of the filesystem that they won’t have being in a snap. The Geany developers warned me about that, but the ones I really wanted appear to work.

Screenshot from 2016-08-30 16-29-54

Posted in Work | Tagged , , , , , , , | 1 Comment

My day of convergence

I’ve had a Nexus 4 since 2013, and I’ve been using it to test out desktop convergence (where you run a desktop environment from the phone) ever since that feature landed just over a year ago. Usually that meant plugging it into my TV via HDMI to make sure it automatically switched to the larger screen, and playing a bit with the traditional windowed-mode of Unity 8, or checking on adaptive layouts in some of the apps. I’ve also run it for hours on end as a demo at conferences such as SCaLE, FOSSETCON, OSCON and SELF. But through all that, I’ve never used it as an actual replacement for my laptop. Until now.

Thanks Frontier

A bit of back-story first. I had been a Verizon FiOS customer for years, and recently they sold all of their FiOS business to Frontier. The transition has been…..less than ideal. A couple of weeks ago I lost all services (phone, TV and internet) and was eventually told that nobody would be out to fix it until the following day. I still had my laptop, but without internet access I couldn’t really do my job on it. And while Ubuntu on phones can offer up a Hotspot, that particular feature doesn’t work on the Nexus 4 (something something, driver, something). Which meant that the only device that I had which could get online was my phone.

No Minecraft for you

13528720_10154238389913419_2608531900571217522_nFortunately, the fact that I’ve been demoing convergence at conferences meant I had all of the equipment I needed to turn my phone into a desktop and keep right on working. I have a bluetooth mouse and keyboard, and a Slimport adapter that let’s me plug it into a bigger screen. But while a TV works for testing, it’s not really great for long-term work. Don’t get me wrong, working from the couch is nice, but the screen is just too far away for reading and writing. Fortunately for me, and unfortunately for my children, their computer is at a desk and is plugged into a monitor with HDMI ports. So I took it over for the day. They didn’t have internet either that day, so they didn’t miss out on much right?

A day of observations

Throughout the day I posted a series of comments on Google+ about my experience. You could go through my post history looking for them, but I’m not going to make you do that. So here’s a quick summary of what I learned:

  • 3G is not nearly fast enough for my daily work. It’s good when using my phone as a phone, doing one thing at a time. But it falls short of broadband when I’ve got a lot of things using it. Still, on that day it was better than my fiber optic service, so there’s that.
  • I had more apps installed on my phone than I thought I did. I was actually taken aback when I opened the Dash in desktop mode and I saw so many icons. It’s far more than I had on Android, though not quite as many as on my laptop.
  • Having a fully-functional Terminal is a lifesaver. I do a lot of my work from the terminal, including IRC, and having one with tabs and keyboard shortcuts for them is a must for me to work.
  • I missed having physical buttons on my keyboard for home/end and page up/down. Thankfully a couple of people came to my rescue in the comments and taught me other combinations to get those.
  • Unity 8 is Unity. Almost all of the keyboard shortcuts that have become second nature to me (an there are a lot of them) were there. There was no learning curve, I didn’t have to change how I did anything or teach myself something new.
  • The phone is still a phone. I got a call (from Frontier, reminding me about an appointment that never happened) while using the device as a desktop. It was a bit disorienting at first, I had forgotten that I was running the desktop the Nexus 4, so when a notification of an incoming call popped up on the screen I didn’t know what was happening. That only lasted a second though, and after clicking answer and picking up the device, I just used it as a phone. Pretty cool

screenshot20160701_151104996

Must go faster

While I was able to do pretty much all of my work that day thanks to my phone, it wasn’t always easy or fun, and I’m not ready to give up my laptop just yet. The Nexus 4 is simply not powerful enough for the kind of workload I was putting on it. But then again, it’s a nearly 4 year old phone, and wasn’t considered a powerhouse even when it was released. The newest Ubuntu phone on the market, the Meizu Pro 5, packs a whole lot more power, and I think it would be able to give a really nice desktop experience.

Posted in Work | Tagged , , , , , | 4 Comments

Dogfooding Unity 8

screenshot20160506_103257823During the Ubuntu Online Summit last week, my colleague Daniel Holbach came up with what he called a “10 day challenge” to some of the engineering manager directing the convergence work in Ubuntu. The idea is simple, try and use only the Unity 8 desktop for 10 working days (two weeks). I thought this was a great way to really identify how close it is to being usable by most Ubuntu users, as well as finding the bugs that cause the most pain in making the switch. So on Friday of last week, with UOS over, I took up the challenge.

Below I will discuss all of the steps that I went through to get it working to my needs. They are not the “official” way of doing it (there isn’t an official way to do all this yet) and they won’t cover every usage scenario, just the ones I faced. If you want to try this challenge yourself they will help you get started. If at any time you get stuck, you can find help in the #ubuntu-unity channel on Freenode, where the developers behind all of these components are very friendly and helpful.

Getting Unity 8

To get started you first need to be on the latest release of Ubuntu. I am using Ubuntu 16.04 (Xenial Xerus), which is the best release for testing Unity 8. You will also need the stable-phone-overlay PPA. Don’t let the name fool you, it’s not just for phones, but it is where you will find the very latest packages for Mir, Unity 8, Libertine and other components you will need. You can install is with this command:

sudo add-apt-repository ppa:ci-train-ppa-service/stable-phone-overlay

Then you will need to install the Unity 8 session package, so that you can select it from the login screen:

sudo apt install unity8-desktop-session

Note: The package above used to be unity8-desktop-session-mir but was renamed to just unity-desktop-session.

When I did this there was a bug in the libhybris package that was causing Mir to try and use some Android stuff, which clearly isn’t available on my laptop. The fix wasn’t yet in the PPA, so I had to take the additional step of installing a fix from our continuous integration system (Note: originally the command below used silo 53, but I’ve been told it is now in silo 31). If you get a black screen when trying to start your Unity 8 session, you probably need this too.

sudo apt-get install phablet-tools phablet-tools-citrain
citrain host-upgrade 031

Note: None of the above paragraph is necessary anymore.

This was enough to get Unity 8 to load for me, but all my apps would crash within a half second of being launched. It turned out to be a problem with the cgroups manager, specifically the cgmanager service was disabled for me (I suspect this was leftover configurations from previous attempts at using Unity 8). After re-enabling it, I was able to log back into Unity 8 and start using apps!

sudo systemctl enable cgmanager

Essential Core Apps

The first thing you’ll notice is that you don’t have many apps available in Unity 8. I had probably more than most, having installed some Ubuntu SDK apps natively on my laptop already. If you haven’t installed the webbrowser-app already, you should. It’s in the Xenial archive and the PPA you added above, so just

sudo apt install webbrowser-app

But that will only get you so far. What you really need are a terminal and file manager. Fortunately those have been created as part of the Core Apps project, you just need to install them. Because the Ubuntu Store wasn’t working for me (see bottom of this post) I had to manually download and install them:

sudo click install --user mhall com.ubuntu.filemanager_0.4.525_multi.click
sudo click install --user mhall com.ubuntu.terminal_0.7.170_multi.click

If you want to use these apps in Unity 7 as well, you have to modify their .desktop files located in ~/.local/share/applications/ and add the -x flag after aa-exec-click, this is because by default it prevents running these apps under X11 where they won’t have the safety of confinement that they get under Mir.

The file manager needed a bit of extra effort to get working. It contains many Samba libraries that allow it to access windows network shares, but for some reason the app was looking for them in the wrong place. As a quick and dirty hack, I ended up copying whatever libraries it needed from /opt/click.ubuntu.com/com.ubuntu.filemanager/current/lib/i386-linux-gnu/ to /usr/lib/i386-linux-gnu/samba/. It’s worth the effort, though, because you need the file manager if you want do things like upload files through the webbrowser.

Using SSH

IRC is a vital communication tool for my job, we all use it every day. In fact, I find it so important that I have a remote client that stays connected 24/7, which I connect to via ssh. Thanks to the Terminal core app, I have quick and easy access to that. But when I first tried to connect to my server, which uses public-key authentication (as they all should), my connection was refused. That is because the Unity 8 session doesn’t run the ssh-agent service on startup. You can start it manually from the terminal:

ssh-agent

This will output some shell commands to setup environment variables, copy those and paste them right back into your terminal to set them. Then you should be able to ssh like normal, and if your key needs a passphrase you will be prompted for it in the terminal rather than in a dialog like you get in Unity 7.

Getting traditional apps

Now that you’ve got some apps running natively on Mir, you probably want to try out support for all of your traditional desktop apps, as you’ve heard advertised. This is done by a project called Libertine, which creates an LXC container and XMir to keep those unconfined apps safely away from your new properly confined setup. The first thing you will need to do is install the libertine packages:

apt-get install libertine libertine-scope

screenshot20160506_105035760Once you have those, you will see a Libertine app in your Apps scope. This is the app that lets you manage your Libertine containers (yes, you can have more than one), and install apps into them. Creating a new container is simply a matter of pressing the “Install” button. You can give it a name of leave it blank to get the default “Xenial”.

screenshot20160506_105618896Once your container is setup, you can install as many apps into it as you want, again using the Libertine container manager. You can even use it to search the archives if you don’t know the exact package name. It will also install any dependencies that package needs into your Libertine container.

screenshot20160506_105942480Now that you have your container setup and apps installed into it, you are ready to start trying them out. For now you have to access them from a separate scope, since the default Apps scope doesn’t look into Libertine containers. That is why you had to install the libertine-scope package above. You can find this scope by clicking on the Dash’s bottom edge indicator to open the Scopes manger, and selecting the Legacy Applications Scope. There you will see launchers for the apps you have installed.

Libertine uses a special container manager to launch apps. If it isn’t running, as was the case for me, your legacy app windows will remain black. To fix that, open up the terminal and manually start the manager:

initctl --session start libertine-lxc-manager

Theming traditional apps

screenshot20160506_122713187By default the legacy apps don’t look very nice. They default to the most basic of themes that look like you’ve time-traveled back to the mid-1990s, and nobody wants to do that. The reason for this is because these apps (or rather, the toolkit they use) expect certain system settings to tell them what theme to use, but those settings aren’t actually a dependency of the application’s package. They are part of a default desktop install, but not part of the default Libertine image.

screenshot20160506_112259969I found a way to fix this, at least for some apps, by installing the light-themes and ubuntu-settings packages into the Libertine container. Specifically it should work for any Gtk3 based application, such as GEdit. It does not, however, work for apps that still use the Gtk2 toolkit, such as Geany. I have not dug deeper to try and figure out how to fix Gtk2 themes, if anybody has a suggestion please leave it in the comments.

What works

It has been a couple of months since I last tried the Unity 8 session, back before I upgraded to Xenial, and at that time there wasn’t much working. I went into this challenge expecting it to be better, but not by much. I honestly didn’t expect to spend even a full day using it. So I was really quite surprised to find that, once I found the workarounds above, I was not only able to spend the full day in it, but I was able to do so quite easily.

screenshot20160509_121832656Whenever you have a new DE (which Unity 8 effectively is) and the latest UI toolkit (Qt 5) you have to be concerned about performance and resource use, and given the bleeding-edge nature of Unity 8 on the desktop, I was expecting to sacrifice some CPU cycles, battery life and RAM. If anything, the opposite was the case. I get at least as many hours on my battery as I do with Unity 7, and I was using less than half the RAM I typically do.

screenshot20160509_103139434Moreover, things that I was expecting to cause me problems surprisingly didn’t. I was able to use Google Hangouts for my video conferences, which I knew had just been enabled in the browser. But I fully expected suspend/resume to have trouble with Mir, given the years I spent fighting it in X11 in the past, but it worked nearly flawlessly (see below). The network indicator had all of my VPN configurations waiting to be used, and they worked perfectly. Even pulse audio was working as well as it did in Unity 7, though this did introduce some problems (again, see below). It even has settings to adjust the mouse speed and disable the trackpad when I’m typing. Most imporantly, nearly all of the keyboard shortcuts that have become subconcious to me in Unity 7 are working in Unity 8.

Most importantly, I was able to write this blog post from Unity 8. That includes taking all of the screenshots and uploading them to WordPress. Switching back and forth between my browser and my notes document to see what I had done over the last few days, or going to the terminal to verify the commands I mentioned above.

What doesn’t

Of course, it wasn’t all unicorns and rainbows, Unity 8 is still very bleeding edge as a desktop shell, and if you want to use it you need to be prepared for some pain. None of it has so far been bad enough to stop me, but your mileage may vary.

One of the first minor pain-points is the fact that middle-click doesn’t paste the active text highlight. I hadn’t realized how much I have become dependent on that until I didn’t have it. You also can’t copy/paste between a Mir and an XMir window, which makes legacy apps somewhat less useful, but that’s on the roadmap to be fixed.

Speaking of windows, Unity 8 is still limited to one per app. This is going to change, but it is the current state of things. This doesn’t matter so much for native apps, which were build under this restriciton, and the terminal app having tabs was a saving grace here. But for legacy apps it presents a bigger issue, especially apps like GTG (Getting Things Gnome) where multi-window is a requirement.

Some power-management is missing too, such as dimming the screen after some amount of inactivity, or turning it off altogether. The session also will not lock when you suspend it, so don’t depend on this in a security-critical way (but really, if you’re running bleeding-edge desktops in security-critical environments, you have bigger problems).

I also had a minor problem with my USB headset. It’s actually a problem I have in Unity 7 too, since upgrading to Xenial the volume and mute controls don’t automatically switch to the headset, even though the audio output and input do. I had a workaround for that in Unity 7, I could open the sound settings and manually change it to the headset, at which point the controls work on it. But in Unity 8’s sound settings there is no such option, so my workaround isn’t available.

The biggest hurdle, from my perspective, was not being able to install apps from the store. This is due to something in the store scope, online accounts, or Ubuntu One, I haven’t figured out which yet. So to install anything, I had to get the .click package and do it manually. But asking around I seem to be the only one having this problem, so those of you who want to try this yourself may not have to worry about that.

The end?

No, not for me. I’m on day 3 of this 10 day challenge, and so far things are going well enough for me to continue. I have been posting regular small updates on Google+, and will keep doing so. If I have enough for a new blog post, I may write another one here, but for the most part keep an eye on my G+ feed. Add your own experiences there, and again join #ubuntu-unity if you get stuck or need help.

Posted in OpenSource, Work | Tagged , , , , , , | 50 Comments

New Ubuntu Community Donations report

Somehow I missed the fact that I never wrote Community Donations report for Q3 2015. I only realized it because it’s time for me to start working on Q4. Sorry for the oversight, but that report is now published.

The next report should be out soon, in the mean time you can look at all of the past reports so see the great things we’ve been able to do with and for the Ubuntu community through this program. Everybody who has recieved these funds have used them to contribute to the project in one way or another, and we appreciate all of their work.

As you may notice, we’ve been regularly paying out more than we’ve been getting in donations. While we’ve had a carry-over balance ever since we started this program, that balance is running down. If you like the things we’ve been able to support with this program, please consider sending it a contribution and helping us spread the word about it.

 

Posted in Community | Tagged , , | 1 Comment

Help make Gnome Software beautiful

As most you you know by now, Ubuntu 16.04 will be dropping the old Ubuntu Software Center in favor of the newer Gnome Software as the graphical front-end to both the Ubuntu archives and 3rd party application store.

Gnome Software

Gnome Software provides a lot of the same enhancements over simple package managers that USC did, and it does this using a new metadata format standard called AppStream. While much of the needed AppStream data can be extracted from the existing packages in the archives, sometimes that’s not sufficient, and that’s when we need people to help fill the gaps.

It turns out that the bulk of the missing or incorrect data is caused by the application icons being used by app packages. While most apps already have an icon, it was never strictly enforced before, and the size and format allowed by the desktop specs was more lenient than what’s needed now.  These lower resolution icons might have been fine for a menu item, but they don’t work very well for a nice, beautiful App Store interface like Gnome Software. And that’s where you can help!

Don’t worry, contributing icons isn’t hard, and it doesn’t require any knowledge of programming or packing to do. Best of all, you’ll not only be helping Ubuntu, but you’ll also be contributing to any other distro that uses the AppStream standard too! In the steps below I will walk you through the process of finding an app in need, getting the correct icon for it, and contributing it to the upstream project and Ubuntu.

1) Pick an App

Because the AppStream data is being automatically extracted from the contents of existing packages, we are able to tell which apps are in need of new icons, and we’ve generated a list of them, sorted by popularity (based on PopCon stats) so you can prioritize your contributions to where they will help the most users. To start working on one, first click the “Create” link to file a new bug report against the package in Ubuntu. Then replace that link in the wiki with a link to your new bug, and put your name in the “Claimed” column so that others know you’ve already started work on it.

Apps with Icon ErrorsNote that a package can contain multiple .desktop files, each of which has it’s own icon, and your bug report will be specific to just that one metadata file. You will also need to be a member of the ~ubuntu-etherpad team (or sub-team like ~ubuntumembers) in order to edit the wiki, you will be asked to verify that membership as part of the login process with Ubuntu SSO.

2) Verify that an AppStream icon is needed

While the extraction process is capable of identifying what packages have a missing or unsupported image in them, it’s not always smart enough to know which packages should have this AppStream data in the first place. So before you get started working on icons, it’s best to first make sure that the metadata file you picked should be part of the AppStream index in the first place.

Because AppStream was designed to be application-centric, the metadata extraction process only looks at those with Type=Application in their .desktop file. It will also ignore any .desktop files with NoDisplay=True in them. If you find a file in the list that shouldn’t be indexed by AppStream, chances are one or both of these values are set incorrectly. In that case you should change your bug description to state that, rather than attaching an icon to it.

3) Contact Upstream

Since there is nothing Ubuntu-specific about AppStream data or icons, you really should be sending your contribution upstream to the originating project. Not only is this best for Ubuntu (carrying patches wastes resources), but it’s just the right thing to do in the open source community. So the after you’ve chosen an app to work on and verfied that it does in fact need a new icon for AppStream, the very next thing you should do is start talking to the upstream project developers.

Start by letting them know that you want to contribute to their project so that it integrates better with AppStream enabled stores (you can reference these Guidelines if they’re not familiar with it), and opening a similar bug report in their bug tracker if they don’t have one already. Finally, be sure to include a link to that upstream bug report in the Ubuntu bug you opened previously so that the Ubuntu developers know the work is also going into upstream to (your contribute might be rejected otherwise).

4) Find or Create an Icon

Chances are the upstream developers already have an icon that meets the AppStream requirements, so ask them about it before trying to find one on your own. If not, look for existing artwork assets that can be used as a logo, and remember that it needs to be at least 64×64 pixels (this is where SVGs are ideal, as they can be exported to any size). Whatever you use, make sure that it matches the application’s current branding, we’re not out to create a new logo for them after all. If you do create a new image file, you will need to make it available under the CC-BY-SA license.

While AppStream only requires a 64×64 pixel image, many desktops (including Unity) will benefit from having even higher resolution icons, and it’s always easier to scale them down than up. So if you have the option, try to provide a 256×256 icon image (or again, just an SVG).

5) Submit your icon

Now that you’ve found (or created) an appropriate icon, it’s time to get it into both the upstream project and Ubuntu. Because each upstream will be different in how they want you to do that, you will need to ask them for guidance (and possibly assistance) in order to do that. Just make sure that you update the upstream bug report with your work, so that the Ubuntu developers can see that it’s been done.

Ubuntu 16.04 has already synced with Debian, so it’s too late for these changes in the upstream project to make their way into this release. In order to get them into 16.04, the Ubuntu packages will have to carry a patch until the changes that land in upstream have the time to make their way into the Ubuntu archives. That’s why it’s so important to get your contribution accepted into the upstream project first, the Ubuntu developers want to know that the patches to their packages will eventually be replaced by the same change from upstream.

attach_file_to_bugTo submit your image to Ubuntu, all you need to do is attach the image file to the bug report you created way back in step #1.

launchpad-subscribeThen, subscribe the “ubuntu-sponsors” team to the bug, these are the Ubuntu developers who will review and apply your icon to the target package, and get it into the Ubuntu archives.

6) Talk about it!

Congratulations, you’ve just made a contribution that is likely to affect millions of people and benefit the entire open source community! That’s something to celebrate, so take to Twitter, Google+, Facebook or your own blog and talk about it. Not only is it good to see people doing these kinds of contributions, it’s also highly motivating to others who might not otherwise get involved. So share your experience, help others who want to do the same, and if you enjoyed it feel free to grab another app from the list and do it again.

Posted in Community, OpenSource, Upstream, Work | Tagged , , , , , , , | 11 Comments