Thursday, February 14, 2019

Testing Voice Assistants 2019

In Facebook memories today, I was reminded of a fun little test I ran 3 years ago comparing the responses of various voice assistants to some possible Valentine's Day queries and interactions. Here were the initial results:

Fun with AI on Valentine's Day: Pt. 2
1) What's a good gift for my boyfriend for Valentine's Day?
Alexa: no response
Siri and Google: list of websites with gift suggestions for guys on Valentine's Day.
2) Where's a good place for dinner on Valentine's Day?
Alexa: no response
Google: lists of websites with recommended restaurants for Valentine's Day in Los Angeles, Minneapolis, Philadelphia, NYC, etc.
Siri: list of 12 restaurants in Billings with reviews on Yelp.
I'd have to say it's a tie between Siri and Google. Siri nailed the local and provided recommended restaurants within a mile of my location. But not all fit with the Valentine's Day theme. I wouldn't pick Pug Mahon's for a romantic dinner. It's not open today. 😉🍺
Google picked up on the romantic theme. If I were a Google exec with a private jet, I'd have a world of possibilities.

And here are the results from 2019:

Happy Valentine's Day
Alexa: Ah, that's sweeter than a box of chocolates. Happy Valentine's Day to you, too.
Google Assistant: Happy Valentine’s Day! If you want to say “I love you” to someone special, you can try saying it in Mandarin. It’s pronounced "wo ai ni" 💗
Siri: Happy Valentine's Day
Cortana: Happy Valentine's Day to you!

What's a good gift for Valentine's Day?
Alexa: Hmm, I don't know that.
Google Assistant: List of websites with gift ideas for Valentine's Day but also included some suggestions for possible search refinements, e.g., What do you get guys for Valentine's Day?
Siri: List of websites with gift ideas for Valentine's Day (those at the top for women. Apparently more people search the web for gifts for women than for men).
Cortana: I'm sorry I can't help out with that one, but I found this on the web. List of websites with one for her followed by one for him. Very egalitarian. 😄

Where should I have dinner on Valentine's Day?
Alexa on Echo Show: 10 Creative Valentine's Day Date Ideas video from USA Today
Google Assistant: Listed 3 nice restaurants within 6 miles.
Siri: List of 5 restaurants within one mile. One was fast food and another, Pug Mahon, was recently closed. 
Cortana: I'm sorry I can't help out with that one, but I found this on the web. List of websites mostly with recipes and a couple with reasons why you should never go out for dinner on Valentine's Day. I guess it answered the question.

Will you be my Valentine?
Alexa: That's sweet to ask but I already have a strong connection to your WiFi.
Google Assistant: I found a few places within 0.8 miles (??)
Siri: I don't have an answer for that
Cortana: Of course I can be your virtual Valentine. Would you like me to sing a love song to celebrate? 
(And goes on to sing, yes sing!)
Red is the rose in yonder garden grows, and fair is the lily of the valley, clear is the water that flows from the Boyne, but my love is fairer than any.

Conclusion: Google Assistant is probably the most efficient at answering questions and providing the kind of information one is looking for but it's lacking in fun.
Alexa is fun to interact with but results differ depending on whether you're interacting with just the voice assistant via Echo or a visual interface on an Echo Show or Spot. I think you'll often get more useful results with a visual interface.
Siri continues to be an also ran. The choice of genders and nationalities offers some personality but it continues to lack the ability to meet what I would consider basic expectations from a voice assistant.
I'd never used Cortana before and was surprised by both the natural voice and quality of responses. But singing a love song verged on creepy to my mind. Flashbacks to "Her".

One additional note: I tested Google Assistant, Siri and Cortana all on an iPad. So, while it is disappointing at first glance that Apple's entry into the AI/voice assistant race is rather lackluster, iOS users do have other options through apps. You can also access Alexa through an iOS app. I'm assuming most of these, aside from Siri, are also available to Android users through the Google Playstore.