Beyond Google Auto

At the past Google IO, Google showed their upcoming version Android N. This version already contains parts that aim at usage in our cars. Previously, Google's efforts towards infotainment systems were bundled in Android Auto. Here, the smartphone had to be connected via USB to the car to use the screen on the dashboard as a screen for your device.

Android Auto (from http://www.digitaltrends.com/infotainment-system-reviews/android-auto-review/)

This worked only if the car's infotainment system supported that. Some tuners, like the Kenwwod DDX9902S, allowed for expanding these capabilities if replacing the current tuner.

Now, this is no longer necessary since the functionality is already built into the OS. This makes it applicable even to older cars. If the car has WLAN as it is available in newer cars, the user also does not need to fiddle around with USB any longer.

Here are some impressions taken from http://www.automotiveitnews.org/articles/share/1481628/

The center of the screens shows the navigation app while the the top and bottom portion of the screen is available for additional information

When initiating a phone call, only the center screen changes

When initiating a phone call, only the center screen changes

While many car manufacturers fear that their brands may become commodity devices, Google already made the next step and provided their answer to the question: Yes, the car will become a cradle for your smartphone.

Google's Offline Personal Assistant

In June 2015, there were some first rumours that few commands for Google Now would be available even when you are offline. An APK teardown of the Google app reported at
http://www.androidpolice.com/2015/06/27/apk-teardown-google-app-v4-8-prepares-for-ok-google-offline-voice-commands-to-control-volume-and-brightness-and-much-more/ revealed some string resources that hinted on this.

<string name="offline_header_text">Offline voice tips</string>
<string name="offline_on_start_cue_cards_header_listening">Offline</string>
<string name="offline_on_start_cue_cards_header_timeout">Offline voice tips</string>
<string name="offline_on_start_cue_cards_second_header_listening">You can still say...</string>
<string name="offline_on_start_cue_cards_second_header_timeout">You can still say "Ok Google," then...</string>
<string name="offline_on_start_cue_cards_second_header_timeout_without_hotword">You can still touch the mic, then say...</string>
<string name="offline_options_start_hotword_disabled">You can still touch the mic, then say...</string>
<string name="offline_options_start_hotword_enabled">You can still say "Ok Google," then...</string>
<string name="offline_error_card_title_text">Something went wrong.</string>
<string name="error_offline_no_connectivity">Check your connection and try again.</string>

So far,  Google Now required an online connection to work. Since this is not always a given it is beneficial to have a workaround in these cases. They found the following four options:

• Make a call
• Send a text
• Play some music
• Turn on Wi-Fi

Usually, such a tear-down is more of the kind of a rumor than facts. In this case, these rumors proved to be true. In September 2015, this functionality was made available as reported, e.g., on http://www.androidpolice.com/2015/09/28/the-google-android-app-now-supports-limited-voice-commands-for-offline-use/. The way this is reflected on the UI is shown on the following picture

Google Now in offline mode,taken from http://www.androidpolice.com/2015/09/28/the-google-android-app-now-supports-limited-voice-commands-for-offline-use/

So, some commands are also available, when you are offline. So far, this list is larger than the original one, but limited to

• Play Music
• Open Gmail (works with any app name on the device)
• Turn on Wi-Fi
• Turn up the volume
• Turn on the flashlight
• Turn on airplane mode
• Turn on Bluetooth
• Dim the screen

Unfortunately, this is only true for the English version. For instance, it refuses to work in German, even if the English offline recognition is downloaded to the device. Neither English or German commands will work. Instead, the following screen is shown.

Google Now in offline mode for German on my Samsung Galaxy 5

Yet, it is unclear, when this will work. But, Google seems to be advancing their embedded technology as reported in Personalized Speech Recognition on Mobile Devices. Here, they describe a remarkable speed-up of their embedded speech recognizer. They state their newest technology "...provides a 2× speed-up in evaluating our acoustic models as compared to the unquantized model, with only a small performance degredation". The recognition performance for open ended dictation in an open domain WER increased from 12.9% to 13.5%. Moreover, they report a decrease of the footprint. Their acoustic model "...is compressed to a tenth of its original size". Apart from that they still feature language model personalization through a combination of vocabulary injection and on-the-fly language model biasing.

In the end, they "built(d) a system which runs 7× faster than real-time on a Nexus 5, with a total system footprint of 20.3 MB"

The latter work only aims for the recognition task as it is available with what they call "Voice Typing". It still needs integration of NLU to make it actual commands to use it for Google Now.

So, Google seems to be on the way for a personal assistant that can also be used if you are not connected to the internet. Some of the commands may make not much sense if you are offline, but some will work, sooner or later. English is supported in first place and it is unclear when other languages will follow. But it is a start.