Finances: Marvin’s first skills

With Marvin’s core architecture designed, I started developing the framework.  The framework itself is intended to handle multiple use cases, from supporting personal information to provide contextual experiences to controlling devices.  Marvin’s key differentiator is that it is powered by data, rather than a simple trigger based on an action.  This is why the core of the framework contains APIs that handle data and extends itself into building out analytics.  Behind the scenes exists an ETL process that feeds into various services, including machine learning.

I’ve been feeding the last 7 years of financial data through to Marvin’s core databases, with just over 9000 transactions. The transactions look something like this:

Date Description Original Description Amount Transaction Type Category Account Name
6/22/2017 Ooma OOMA, INC 08887116662 CA XX.XX debit Home Phone Smart Cash Platinum Plus MasterCard
6/22/2017 Costco COSTCO WHOLESALE W159 AJAX ON XX.XX debit Home Supplies Smart Cash Platinum Plus MasterCard
6/21/2017 Transfer to Chequing TRANSFER OUT XX.XX debit Transfer General Savings
6/21/2017 Transfer from General Savings TRANSFER IN XX.XX credit Transfer Chequing
6/20/2017 Costco COSTCO WHOLESALE W1128 OSHAWA ON XX.XX debit Groceries Smart Cash Platinum Plus MasterCard
6/20/2017 Costco WWW COSTCO CA 905-264-8337 ON XX.XX debit Sporting Goods Smart Cash Platinum Plus MasterCard
6/20/2017 Taunton Endo TAUNTON ENDO OSHAWA ON XX.XX debit Doctor Smart Cash Platinum Plus MasterCard

These transactions provide a very good base for generating some initial learning models. By including credit or debit, dates, retailers, amounts, categories for purchases and labels, it enables various types of skills to be identified.  These skills help me understand my finances better while also helping me improve.

Continue reading…

Disruptive Companies, Passionate Teams

I’ve come to the realization that I like companies that try to disrupt a market. Whether it be phones, IoT devices, or software. Love rooting for the next up and coming success story, even if it means some growing pains along the way. I tend to be a very passionate person when it comes to technology.  I find myself pouring over details, specs, and features with each new thing. It’s rewarding to engage their teams over the details, as you get to see their drive and dedication.

The products trying to disrupt the market are often ones that bring in their own special flare. Even though they may not offer all the same features as the industry leaders, you can tell they are making a difference through the details. The communications coming out of these teams show their passion.  They don’t often generate hostility. As a result, their message tends to be softer and they normally try to rally others to join them.  As a result of a more neutral tone, hoping to rally people together, you often find that these teams often offer the best support. With good support comes a good and lasting impression on the market, because the best form of marketing is word of mouth.

Because of my passion for technology and great products, I often find myself going out of my way to find these new and upcoming products or technologies.  Over the years, I’ve bought into or crowdfunded numerous products that were either help achieve a personal or professional need.  I don’t always buy into products solely for the product itself, I often do it for the experience and the support. I’ve had the opportunity to follow a number of companies who were just starting out to flourishing into companies now competing against some industry leaders or going through large acquisitions. These teams were huge on gathering feedback and interacting with their communities or “evangelists”.

Continue reading…

Should Products Ensure Developer Happiness?

Over the past while, I’ve been fully immersed these days into figuring out how to build Marvin.  In doing so, I’ve been researching products, services, or even technology that would help me do it. While diving into each one and evaluating whether to proceed in using them, I’ve found myself looking for the very same thing in each one.  That is, are they working to ensure their developers are happy?  Would I be happy working with them?

One of the biggest gaps that I’ve been looking to fill is the financial transactional information.  To do that, I’ve been evaluating Plaid, Xignite, Yodlee, and Finicity.  All are very capable and established services.  However, of the 4 services, Plaid is the only one that makes me feel that they care about developers. They have a healthy set of SDKs, great documentation, and their dev portal is welcoming.

Companies from all over are beefing up their engineering and development teams.  In many of those technology focused companies, the decision makers are those who are working directly with the product or service, the developers.  Developers normally choose the product that is the most effective at helping them achieve their goals.  Many products today continue to focus on selling to the business minded person, focusing on the features and pricing.  While that is certainly not a bad strategy, they tend to completely avoid the developers who would be working with the product or service.

I believe we are reaching the tipping point where you need to appealing to developers and their happiness in order to succeed. Many products brand themselves as a platform, normally with some externally facing API.  Those that don’t have an API likely have it near the top of their roadmap.  The API can serve multiple purposes: to bring a richer experience, to share data with other tools, or to build new functionality which isn’t supported within the parameters of the service. The market is flooded with technology companies, each trying to cater to a niche. While that company is using your product or service, their goals may be different, which is why they look for API access to enable them to perform their own business logic.

Continue reading…

Marvin – Personalized and Smart Virtual Assistant

Over the last two years, I’ve been buying into Internet of Things (IoT) devices. While the devices themselves may be smart, they do not make a smart home.  In comes Marvin.  Unlike many who have gone for Jarvis as a name, I chose the name Marvin from Hitchhiker’s Guild to the Galaxy. Marvin has a brain the size of a planet, which is fitting considering the information it will be processing.
Continue reading…

The rise of NoSQL is an opportunity for new RDBMS solutions

It should come as no surprise that NoSQL has become popular over the past few years. This popularity has been driven in large part by the app revolution. Many new apps are hitting millions of users in less than a week, some in a day. This presents a scaling problem for app developers who seek a large audience.

Scaling a typical RDBMS system like MySQL or MsSQL from 0 to 1 million users has never been easy.  You have to setup master and slave servers, shard and balance the data, and ensure you have resources in place for any unexpected events. NoSQL is being touted as the solution to those problems. It shouldn’t be.

NoSQL’s use cases have been mistakenly focused on scalability because of the complexities of putting up an RDBMS and scaling it. Application developers aren’t necessarily interested in becoming server side wizards.  They prefer to focus on building out their apps, not scaling the servers powering those apps.  These developers are looking for something low cost that can keep up with their needs as they grow. Developers have flocked to NoSQL for these very reasons.

However, when developers look to grow their apps and introduce more complex functionality, they sometimes hit roadblocks due to the way data is stored in NoSQL.  A solution to one of those roadblocks is MapReduce.  MapReduce brings index like functionality to NoSQL.

My goal isn’t to dispute the importance of NoSQL, but to promote the reality that not all database systems are alike -many serve their specific purposes. NoSQL’s benefits are to bring terrific key-value access with great performance.  To some the lack of schema is a benefit that allows the application to control how the data is stored, limiting the need to interface with and configure the database.

Over the years I’ve been looking to put up products that need RDBMS like storage that scales. NoSQL just couldn’t do it for me.  Many would agree, but do not know of anything better. Lucky for me, I found VoltDB. To this day, sales people continue to contact me to use their NoSQL solution. I ask: How can NoSQL solve my problems? Are you ACID compliant? How can I merge data from multiple tables? How can I use my data to build out analytics? Most of the time, the sales teams can only sell me on one problem, scaling. They often forget that I have to sacrifice functionality to solve my scaling problems. One should never have to compromise.

If NoSQL is known for scale, how well does VoltDB do it? A picture is worth a thousand, or in this case, a million words.

VoltDB benchmarks from May 2013.

The chart above shows VoltDB achieving 1m transactions per second (it’s a 1 year-old benchmark). Place this benchmark next to the top NoSQL solutions and you will find yourself with an equal or better performing solution.  Best of all, VoltDB does it without sacrificing the common features we need from relational database systems.

The switch from a traditional database like MySQL or MsSQL to VoltDB is simple and can often be measured in hours or days.  A switch from a traditional RDBMS to NoSQL on the other hand is likely to be measured in weeks, days if you are lucky.

VoltDB is a NewSQL solution. NewSQL, a term used to avoid the poor scaling stigmata attributed to RDBMS or typical SQL solutions.

NewSQL’s solution solves today’s data problems without creating new complexities.  Ever heard of trying to fit a square peg in a round hole? NoSQL is that square peg, and it is doing its best to go through that round hole by solving scaling problems with a different solution, causing new complexities. Complexities that arise when trying pulling complex data sets for analytic purposes, adding BI support, updating schema, or normalizing data.

Many of today’s biggest companies, including Twitter use NoSQL systems. Had NewSQL been around when they had issues scaling would the problem still be solved with NoSQL? Chances are NewSQL would also be their solution. NewSQL builds on the decades of research and innovation attributed to relational databases. They have matured and have been solving many of the world’s most complex data problems.

In case this argument for NewSQL doesn’t quite bring it home for you, I will be writing another article supported by detailed use cases. Until then, please let me know what you think here or on Twitter @francispelland.

What about these electric cars? Are they really worth the hype?

Following up on my last post, which was more focused around Google and the driverless car comes this post. Tesla too, created by a very smart and ambitious Elon Musk. Not that I am harping on the accomplishments or the vision, however a lot of what comes out of Tesla in my opinion seem to be repeating the past.  We all know about “Who Killed the Electric Car”.  Has Tesla learned from that? Are they doing things differently?

In my opinion, of which has been formulated based on facts, is that Tesla is repeating some of the same mistakes as GM did in the 90s.  Electric cars carry something called range anxiety.  Replacement batteries are expensive. Charging your battery takes much longer than filling up your tank with gas. These aren’t easy problems to address, but ones that require a lot of money and effort to solve.

Continue reading…

Why is the tech industry desperate to disrupt the auto industry if it doesn’t know what is all about?

Technology blogs and journalists are praising the likes of Google, Tesla and many others.  This is great, those companies are creating terrific products. They are innovating in a market that has been relatively slow to turn around and that has been plagued with various degrees of problems. So why you wonder am I writing this after the title I put up? The short answer… Google, Tesla, and others are NOT the solution or the disruptors of the auto industry.

I’ve gone off on several rants on Facebook, Google+ and in some cases on Twitter.  I really do not want to discredit the advancements Google has made on self driving cars. I also am not looking to discredit Tesla for releasing an electric car. But those products are only scratching the surface of what a full car that can fully disrupt the industry should be. I know, you are probably saying “well Tesla is selling cars”. Yep, you are right. They are good cars. But let me ask you this, are they available to anyone? Is it an inconvenience for you to buy one?

Continue reading…

Contextual Series: User Engagement

In the second part of the contextual series, I will be continuing from where I left off last time with User Onboarding. I’ll be covering how to make your application relevant after the user has gotten past their initial experience.  The goal is to engage with the user in a way that will retain them and have them spend money.

The first few minutes of the user trying your product is crucial. In that short period of time they will decide whether they will uninstall it and move on or keep trying it out. But the next 30 minutes are also just as important. Like a drug, you want the user hooked to your product, you want them to feel as though they are dependant of it.  The most common way products get you addicted is through social engineering by getting you to engage with people you know. While it is something I would recommend each product would do, the product should be able to stand on it’s own even if the user has no friends.

Let’s assume that your product is capable of tracking a lot of data, including basic user profiles to narrow down their demographics and activity history. The data can be used to improve your analytics, determine business logic, and can be used to feed into your contextual engines.

Continue reading…

Is Lunarpages storing passwords in plaintext?

First of all, I want to state that these are serious claims. That what I am stating is based on my observations.  While I cannot confirm what I am saying is true, it is hard to claim the situation is much different than what I am stating.

Lunarpages, as I wrote in my previous post, has been my hosting provider for nearly 10 years. I’ve grown up from being that wacky teenager with limited technological understanding to now architecting and building infrastructure and development tools. I don’t claim to be an expert in any way and there is a lot I can learn.

Over the past year, I noticed some very shady things going on with my account. In part, my account was compromised at least twice where files were overridden. To this day, I still do not know how it has happened. But lately having to deal with support, I noticed a rather major flaw.  Lunarpages and it’s staff have access to your passwords in plain text.

To come up with the conclusion that Lunarpages in has access to your passwords in plain text comes from two separate emails I’ve received from them in the past week.  Frankly, I have lost all faith in them as a company.

Continue reading…