Make Android apps Human with NDR

Bicentennial Man PosterEver since we started building the Ubuntu SDK, we’ve been trying to find ways of bringing the vast number of Android apps that exist over to Ubuntu. As with any new platform, there’s a chasm between Android apps and native apps that can only be crossed through the effort of porting.

There are simple solutions, of course, like providing an Android runtime on Ubuntu. On other platforms, those have shown to present Android apps as second-class citizens that can’t benefit from a new platform’s unique features. Worse, they don’t provide a way for apps to gradually become first-class citizens, so chasm between Android and native still exists, which means the vast majority of apps supported this way will never improve.

There are also complicates solutions, like code conversion, that try to translate Android/Java code into the native platform’s language and toolkit, preserving logic and structure along the way. But doing this right becomes such a monumental task that making a tool to do it is virtually impossible, and the amount of cleanup and checking needed to be done by an actual developer quickly rises to the same level of effort as a manual port would have. This approach also fails to take advantage of differences in the platforms, and will re-create the old way of doing things even when it doesn’t make sense on the new platform.

Screenshot from 2014-04-19 14:44:22NDR takes a different approach to these, it doesn’t let you run our Android code on Ubuntu, nor does it try to convert your Android code to native code. Instead NDR will re-create the general framework of your Android app as a native Ubuntu app, converting Activities to Pages, for example, to give you a skeleton project on which you can build your port. It won’t get you over the chasm, but it’ll show you the path to take and give you a head start on it. You will just need to fill it in with the logic code to make it behave like your Android app. NDR won’t provide any of logic for you, and chances are you’ll want to do it slightly differently than you did in Android anyway, due to the differences between the two platforms.

Screenshot from 2014-04-19 14:44:31To test NDR during development, I chose the Telegram app because it was open source, popular, and largely used Android’s layout definitions and components. NDR will be less useful against apps such as games, that use their own UI components and draw directly to a canvas, but it’s pretty good at converting apps that use Android’s components and UI builder.

After only a couple days of hacking I was able to get NDR to generate enough of an Ubuntu SDK application that, with a little bit of manual cleanup, it was recognizably similar to the Android app’s.

This proves, in my opinion, that bootstrapping an Ubuntu port based on Android source code is not only possible, but is a viable way of supporting Android app developers who want to cross that chasm and target their apps for Ubuntu as well. I hope it will open the door for high-quality, native Ubuntu app ports from the Android ecosystem.  There is still much more NDR can do to make this easier, and having people with more Android experience than me (that would be none) would certainly make it a more powerful tool, so I’m making it a public, open source project on Launchpad and am inviting anybody who has an interest in this to help me improve it.

Posted in OpenSource, Programming, Projects, Work | Tagged , , , , , , , , , , | 2 Comments

My phone is lonely, let’s fix that

I’ve been using Ubuntu on my only phone for over six months now, and I’ve been loving it. But all this time it’s been missing something, something I couldn’t quite put my finger on. Then, Saturday night, it finally hit me, it’s missing the community.

That’s not to say that the community isn’t involved in building it, all of the core apps have been community developed, as have several parts of our toolkit and even the platform itself. Everything about Ubuntu for phones is open source and open to the community.

But the community wasn’t on my phone. Their work was, but not the people.  I have Facebook and Google+ and Twitter, sure, but everybody is on those, and you have to either follow or friend people there to see anything from them. I wanted something that put the community of Ubuntu phone users, on my Ubuntu phone. So, I started to make one.

Community Cast

Community Cast is a very simple, very basic, public message broadcasting service for Ubuntu. It’s not instant messaging, or social networking. It doesn’t to chat rooms or groups. It isn’t secure, at all.  It does just one thing, it lets you send a short message to everybody else who uses it. It’s a place to say hello to other users of Ubuntu phone (or tablet).  That’s it, that’s all.

As I mentioned at the start, I only realized what I wanted Saturday night, but after spending just a few hours on it, I’ve managed to get a barely functional client and server, which I’m making available now to anybody who wants to help build it.

Server

The server piece is a very small Django app, with a single BroadcastMessage data model, and the Django Rest Framework that allows you to list and post messages via JSON. To keep things simple, it doesn’t do any authentication yet, so it’s certainly not ready for any kind of production use.  I would like it to get Ubuntu One authentication information from the client, but I’m still working out how to do that.  I threw this very basic server up on our internal testing OpenStack cloud already, but it’s running the built-in http server and an sqlite3 database, so if it slows to a crawl or stops working don’t be surprised.  Like I said, it’s not production ready.  But if you want to help me get it there, you can get the code with bzr branch lp:~mhall119/+junk/communitycast-server, then just run syncdb and runserver to start it.

Client

The client is just as simple and unfinished as the server (I’ve only put a few hours into them both combined, remember?), but it’s enough to use. Again there’s no authentication, so anybody with the client code can post to my server, but I want to use the Ubuntu Online Accounts to authenticate a user via their Ubuntu One account. There’s also no automatic updating, you have to press the refresh button in the toolbar to check for new messages. But it works. You can get the code for it with bzr branch lp:~mhall119/+junk/communitycast-client and it will by default connect to my test instance.  If you want to run your own server, you can change the baseUrl property on the MessageListModel to point to your local (or remote) server.

Screenshots

There isn’t much to show, but here’s what it looks like right now.  I hope that there’s enough interest from others to get some better designs for the client and help implementing them and filling out the rest of the features on both the client and server.

communitycast-client-1communitycast-client-2communitycast-client-3

Not bad for a few hours of work.  I have a functional client and server, with the server even deployed to the cloud. Developing for Ubuntu is proving to be extremely fast and easy.

 

Posted in OpenSource, Programming, Projects | Tagged , , , , , , , , , , | 5 Comments

First steps towards a converged email client for Ubuntu

Screenshot from 2014-03-20 21:57:06Yesterday we made a big step towards developing a native email client for Ubuntu, which uses the Ubuntu UI Toolkit and will converge between between phones, tablets and the desktop from the start.

We’re not starting from scratch though, we’re building on top of the incredible work done in the Trojitá project.  Trojitá provides a fast, light email client built with Qt, which made it ideal for using with Ubuntu. And yesterday, the first of that work was accepted into upstream, you can now build an Ubuntu Components front end to Trojitá.

None of this would have been possible without the help up Trojitá’s upstream developer Jan Kundrát, who patiently helped me learn the codebase, and also the basics of CMake and Git so that I could make this first contribution. It also wouldn’t have been possible without the existing work by Ken VanDine and Joseph Mills, who both worked on the build configuration and some initial QML code that I used. Thanks also to Dan Chapman for working together with me to get this contribution into shape and accepted upstream.

This is just the start, now comes the hard work of actually building the new UI with the Ubuntu UI Toolkit.  Andrea Del Sarto has provided some fantastic UI mockups already which we can use as a start, but there’s still a need for a more detailed visual and UX design.  If you want to be part of that work, I’ve documented how to get the code and how to contribute on the EmailClient wiki.  You can also join the next IRC meeting at 1400 UTC today in #ubuntu-touch-meeting on Freenode.

Posted in OpenSource, Programming, Projects, Work | Tagged , , , , , , , , , , | 2 Comments

Work begins on Qimo 3.0

Today I started work on the third release of Qimo.  I’m slowly but surely learning how to do it better.

In the first release of Qimo I unpacked an Ubuntu ISO by hand, changed things by hand, and then re-packed the ISO….by hand. It was really quite a bit of work.

By the second release I had learned to build debian packages to install Qimo’s artwork, configurations and dependencies.  This make things a good bit easier, but I was still unpacking Ubuntu’s ISO, ripping things out, installing my new packages, hacking a few more things together, and then packing it up again.

Now, for the third release, I’m learning how to use debootstrap to create a brand new ISO, just for Qimo, with just the things that Qimo needs to run. I’m still learning how it all works, but I’m hoping it will reduce the time and effort it takes to spin up a new CD image, which will let me iterate faster on the actual development of Qimo, and maybe even provide regular image downloads during development.

Who knows, if all goes well, I might even have time to fix up the website.

Posted in OpenSource, Programming | Tagged , , | 3 Comments

Ubuntu App Developer Week starts Today!

Starting at 1400 UTC today, and continuing all week long, we will be hosting a series of online classes covering many aspects of Ubuntu application development. We have experts both from Canonical and our always amazing community who will be discussing the Ubuntu SDK, QML and HTML5 development, as well as the new Click packaging and app store.

You can find the full schedule here: http://summit.ubuntu.com/appdevweek-1403/

We’re using a new format for this year’s app developer week.  As you can tell from the link above, we’re using the Summit website.  It will work much like the virtual UDS, where each session will have a page containing an embedded YouTube video that will stream the presenter’s hangout, an embedded IRC chat window that will log you into the correct channel, and an Etherpad document where the presenter can post code examples, notes, or any other text.

Use the chatroom like you would an Ubuntu On Air session, start your questions with “QUESTION:” and wait for the presenter to get to it. After the session is over, the recorded video will be available on that page for you to replay later. If you register yourself as attending on the website (requires a Launchpad profile), you can mark yourself as attending those sessions you are interested in, and Summit can then give you a personalize schedule as well as an ical feed you can subscribe to in your calendar.

If you want to use the embedded Etherpad, make sure you’re a member of https://launchpad.net/~ubuntu-etherpad

That’s it!  Enjoy the session, ask good questions, help others when you can, and happy hacking.

Posted in Events, OpenSource, Work | Tagged , , , , , , , | Leave a comment

UbBloPoMo ends

I almost forgot to post today, which would have been sad because it’s the last day of my UbBloPoMo plan. But I still have 30 minutes before March 1st (my time), so I’ll type quickly to make this one count.

This will be the 21st post using the UbBloPoMo tag, and I’m quite pleased with the way it turned out. Not only has it gotten me out of a rut on not blogging, it also boosted my site traffic rather significantly and, I hope, given Planet Ubuntu readers a little more regular and irregular content.

I did miss one day, but the following post proved to be the most popular of the month.  It got a lot of people talking and even landed me a guest spot on the Linux Unplugged show, which was quite a bit of fun and I would love to do again (subtle hints there).

I don’t plan on continuing this blog-a-day through into March, but I do plan on keeping this blog more active than it was most of last year.  I hope you’ve all enjoyed reading these posts as much as I’ve enjoyed writing them, and even more I hope I’ve inspired some of you to blog more on your own websites.

Posted with 10 minutes to spare.

Posted in OpenSource, Projects | Tagged , , | Leave a comment

My App Showdown Wishlist

Today we announced the start of the next Ubuntu App Showdown, and I have very high hopes for the kinds of apps we’ll see this time around. Our SDK has grown by leaps and bounds since the last one, and so much more is possible now. So go get yourself started now: http://developer.ubuntu.com/apps/

Earlier today Jono posted his Top 5 Dream Ubuntu Apps, and they all sound great.  I don’t have any specific apps I’d like to see, but I would love to get some multi-player games.  Nothing fancy, nothing 3D or FPS.  Think more like Draw Something or Words With Friends, something casual, turn-based, that lets me connect with other Ubuntu device users. A clone of one of those would be fun, but let’s try and come up with something original, something unique to Ubuntu.

What do you say, got any good ideas?  If you do, post them in the App Showdown subreddit or our Google+ App Developers community and let’s make it happen.

Posted in OpenSource, Programming, Projects | Tagged , , , , , , , | 3 Comments

New roof, new developer portal, new scopes, what a week!

It’s been a crazy busy week, and it’s only Tuesday (as of this writing)!  Because I’m exhausted, this is going to be a short post listing the things that are new.

New Roof

I wrote earlierthat I was having a new roof put on my house.  Well that all starter unceremoniously at 7:30am on Monday, and the hammering over my head has been going on non-stop for two full working days.  Everybody who joined me on a Google+ Hangout has been regaled with the sounds of my torment.  It looks nice though, so there’s that.

New Developer Portal

Well, new-ish.  We heavily revamped the Apps section to include more walk-through content to help new Ubuntu app developers learn the tools, the process and the platform.  If you haven’t been there yet, you really should give it a read and get yourself started: http://developer.ubuntu.com/apps/

New HTML5 APIs

In addition to the developer portal itself, I was able to publish new HTML5 API docs for the 14.04 release of Ubuntu.  Not only does this include the UbuntuUI library from the previous release, it also introduced new platform APIs for Content Hub, Online Accounts and Alarms, with more platform APIs coming soon.  The Cordova 3.4 API docs are proving harder to parse and upload than I anticipated, but I will hopefully have them published soon. If you’re an HTML5 app developer, you’ll be interested in these: http://developer.ubuntu.com/api/html5/sdk-14.04/

New Scopes

While not exactly a secret, we did start to make some noise about the new Scopes framework and Unity Dash that bring in a lot of improvements. As much as I liked the Home lens searching everything and aggregating results, it just wasn’t reaching the potential we had hoped for it.  The new setup will allow scopes to add more information that is specific to their result types, control how those results are displayed, and more clearly brand themselves to let the user know what’s being searched. You can read more about the enhancements at http://developer.ubuntu.com/2014/02/introducing-our-new-scopes-technology/ Like I said, it’s been a crazy busy week.  And we’re not done yet!

Posted in Programming | Tagged , , , , , , , | Leave a comment

There is no “Touch”, only “Ubuntu”

There’s been a lot of talk about Ubuntu’s phone and tablet development over the last year, and it’s great that it’s getting so much attention, but people have been getting the name of it all wrong. Now, to be fair, this is a problem entirely of our own making, we started off talking about the phone (and later tablet) developments as “Ubuntu Touch”, and put most of the information about on our wiki under a page named Touch.  But there is no Ubuntu Touch! It’s not a separate OS or platform, there is only one OS and it’s simply called Ubuntu.

Ubuntu 14.04 Stack

What people are referring to when they say Touch or Ubuntu Touch, is really just Ubuntu with Unity 8.  Other than the shell (and display server that powers it), it’s the same OS as you get on your desktop.

Everything under the hood is the same: same tools, same filesystem, even the same version of them, because it’s all built from the same source. Calendar data is stored in the same place, audio and video is played through the same system, even the Unity APIs are shared between desktop and phone.

So why is the name important?  Not only is it more accurate to call them both Ubuntu, it’s also one of the (in my opinion) most exciting things about having an Ubuntu phone.  You’re not getting a stripped down embedded Linux OS, or something so customized for phones that it’s useless on your desktop.  You’re getting a fully featured, universal operating system, one that can do everything you need from a phone and everything you need from a desktop.

Future Ubuntu Stack

This is the key to Ubuntu’s convergence strategy, something that nobody else has right now. Android makes a terrible desktop OS.  So does iOS.  Chrome OS won’t work for a phone either, nor OSX. Even Microsoft has built two different platforms for mobile and desktop, even if they’ve slapped the same interface on both.

But with Ubuntu, once Unity 8 comes to the desktop, you will have the same OS, the same platform, on all of your devices. And while you will run the same version of Unity on both, Unity 8 is smart enough to change how it looks and how it works to meet the needs and capabilities of what you’re running it on.  Better still, Unity will be able to make these changes at run time, so if you dock your convertible tablet to a keyboard, it will automatically switch from giving you a tablet interface to a desktop interface. All of your running apps keep running, but thanks to the Ubuntu SDK those too will automatically adjust to work as desktop apps.

So while “Ubuntu Touch” may have been a useful distinction in the beginning, it isn’t anymore.  Instead, if you need to differentiate between desktop and mobile versions of Ubuntu, you should refer to “Unity 8″ if talking about the interface, or “Ubuntu for phones” (or tablet) if you’re talking about device images or hardware enablement. And if you’re a developer and you are talking about the platform APIs or capabilities, you’re talking about the “Ubuntu SDK”, which is already available on both desktop and mobile installs of Ubuntu.

Posted in OpenSource | Tagged , , , , , , , | 17 Comments

Ubuntu API Website Teardown

Ubuntu API Website

For much of the past year I’ve been working on the Ubuntu API Website, a Django project for hosting all of the API documentation for the Ubuntu SDK, covering a variety of languages, toolkits and libraries.  It’s been a lot of work for just one person, to make it really awesome I’m going to need help from you guys and gals in the community.

To help smooth the onramp to getting started, here is a breakdown of the different components in the site and how they all fit together.  You should grab a copy of the branch from Launchpad so you can follow along by running: bzr branch lp:ubuntu-api-website

Django

First off, let’s talk about the framework.  The API website uses Django, a very popular Python webapp framework that’s also used by other community-run Ubuntu websites, such as Summit and the LoCo Team Portal, which makes it a good fit. A Django project consists of one or more Django “apps”, which I will cover below.  Each app consists of “models”, which use the Django ORM (Object-Relational Mapping) to handle all of the database interactions for us, so we can stick to just Python and not worry about SQL.  Apps also have “views”, which are classes or functions that are called when a URL is requested.  Finally, Django provides a default templating engine that views can use to produce HTML.

If you’re not familiar with Django already, you should take the online Tutorial.  It only takes about an hour to go through it all, and by the end you’ll have learned all of the fundamental things about building a Django site.

Branch Root

When you first get the branch you’ll see one folder and a handful of files.  The folder, developer_network, is the Django project root, inside there is all of the source code for the website.  Most of your time is going to be spent in there.

Also in the branch root you’ll find some files that are used for managing the project itself. Most important of these is the README file, which gives step by step instructions for getting it running on your machine. You will want to follow these instructions before you start changing code. Among the instructions is using the requirements.txt file, also in the branch root, to setup a virtualenv environment.  Virtualenv lets you create a Python runtime specifically for this project, without it conflicting with your system-wide Python installation.

The other files you can ignore for now, they’re used for packaging and deploying the site, you won’t need them during development.

./developer_network/

As I mentioned above, this folder is the Django project root.  It has sub-folders for each of the Django apps used by this project. I will go into more detail on each of these apps below.

This folder also contains three important files for Django: manage.py, urls.py and settings.py

manage.py is used for a number of commands you can give to Django.  In the README you’ll have seen it used to call syncdbmigrate and initdb.  These create the database tables, apply any table schema changes, and load them with initial data. These commands only need to be run once.  It also has you run collectstatic and runserver. The first collects static files (images, css, javascript, etc) from all of the apps and puts them all into a single ./static/ folder in the project root, you’ll need to run that whenever you change one of those files in an app.  The second, runserver, runs a local HTTP server for your app, this is very handy during development when you don’t want to be bothered with a full Apache server. You can run this anytime you want to see your site “live”.

settings.py contains all of the Django configuration for the project.  There’s too much to go into detail on here, and you’ll rarely need to touch it anyway.

urls.py is the file that maps URLs to an application’s views, it’s basically a list of regular-expressions that try to match the requested URL, and a python function or class to call for that match. If you took the Django project tutorial I recommended above, you should have a pretty good understanding of what it does. If you ever add a new view, you’ll need to add a corresponding line to this file in order for Django to know about it. If you want to know what view handles a given URL, you can just look it up here.

./developer_network/ubuntu_website/

If you followed the README in the branch root, the first thing it has you do is grab another bzr branch and put it in ./developer_network/ubuntu_website.  This is a Django app that does nothing more than provide a base template for all of your project’s pages. It’s generic enough to be used by other Django-powered websites, so it’s kept in a separate branch that each one can pull from.  It’s rare that you’ll need to make changes in here, but if you do just remember that you need to push you changes branch to the ubuntu-community-webthemes project on Launchpad.

./developer_network/rest_framework/

This is a 3rd party Django app that provides the RESTful JSON API for the site. You should not make changes to this app, since that would put us out of sync with the upstream code, and would make it difficult to pull in updates from them in the future.  All of the code specific to the Ubuntu API Website’s services are in the developer_network/service/ app.

./developer_network/search/

This app isn’t being used yet, but it is intended for giving better search functionality to the site. There are some models here already, but nothing that is being used.  So if searching is your thing, this is the app you’ll want to work in.

./developer_network/related/

This is another app that isn’t being used yet, but is intended to allow users to link additional content to the API documentation. This is one of the major goals of the site, and a relatively easy area to get started contributing. There are already models defined for code snippets, Images and links. Snippets and Links should be relatively straightforward to implement. Images will be a little harder, because the site runs on multiple instances in the cloud, and each instance will need access to the image, so we can’t just use the Django default of saving them to local files. This is the best place for you to make an impact on the site.

./developer_network/common/

The common app provides views for logging in and out of the app, as well as views for handling 404 and 500 errors when the arise.  It also provides some base models the site’s page hierarchy. This starts with a Topic at the top, which would be qml or html5 in our site, followed by a Version which lets us host different sets of docs for the different supported releases of Ubuntu. Finally each set of docs is placed within a Section, such as Graphical Interface or Platform Service to help the user browse them based on use.

./developer_network/apidocs/

This app provides models that correspond directly to pieces of documentation that are being imported.  Documentation can be imported either as an Element that represents a specific part of the API, such as a class or function, or as a Page that represents long-form text on how to use the Elements themselves.  Each one of these may also have a given Namespace attached to it, if the imported language supports it, to further categorize them.

./developer_network/web/

Finally we get into the app that is actually generates the pages.  This app has no models, but uses the ones defined in the common and apidocs apps.  This app defines all of the views and templates used by the website’s pages, so no matter what you are working on there’s a good chance you’ll need to make changes in here too. The templates defined here use the ones in ubuntu_website as a base, and then add site and page specific markup for each.

Getting Started

If you’re still reading this far down, congratulations! You have all the information you need to dive in and start turning a boring but functional website into a dynamic, collaborative information hub for Ubuntu app developers. But you don’t need to go it alone, I’m on IRC all the time, so come find me (mhall119) in #ubuntu-website or #ubuntu-app-devel on Freenode and let me know where you want to start. If you don’t do IRC, leave a comment below and I’ll respond to it. And of course you can find the project, file bugs (or pick bugs to fix) and get the code all from the Launchpad project.

Posted in OpenSource, Programming, Projects, Work | Tagged , , , , , , , , , | 1 Comment