Category Archives: Open Source

Exploring Udacity’s 1st 40GB driving data set

I read about the release of their second data set yesterday and wanted to check it out.  For convenience, I downloaded the original, smaller, data set.

Preface: ROS is only officially supported on Ubuntu & Debian and is experimental on  OS X (Homebrew), Gentoo, and OpenEmbedded/Yocto.

Getting the data

Download the data yourself: Driving Data

The data set, which is linked to from the page above, was served up from Amazon S3 and actually seemed quite slow to download, so I let it run late last night and started exploring today.

The compressed download is dataset.bag.tar.gz

20161006-udacity-dataset-bag

and after extracting is a 42.3 GB file dataset.bag

.bag is a file type associated with the Robot Operating System (ROS)

Data overview

To get an overview of the file use the rosbag info <filename> command:

result

Open in new window

There are 28 data topics from on-board sensors including 3 color cameras.  Topics:

  • /center_camera/image_color
  • /left_camera/image_color
  • /right_camera/image_colors

Each camera topic has 15212 messages.   Doing the math on 15212 messages / 760 seconds works out to roughly 20 frames per second.

Viewing the video streams

Converting a camera topic to a standalone video is a two step process:

  1. export jpegs from the bag file
  2. convert the jpegs to video

Exporting the jpegs

To export the image topic to jpegs, the bag needs to be played back and the frames extracted.  This can be done with a launch script.  The default filename pattern (frame%04d.jpg) allows for 4 numerical figures, so we need to add the following line to modify the default file name pattern into one that allows for 5 digits:

The entire script below that launches the player and extractor:

The number of resulting frames should match the number of topic messages seen from info.

If not, as was our case, the default sec per frame time should be changed.  It seems counter-intuitive, but after slowing down the rate, trying “0.11” and “0.2”, the number of frames extracted was also going down.  I settled on “0.02” seconds per frame which resulted in the correct number of frames.  Add the line to the launch script.

The working launch script now looks like this:

Download working Left, Center, and Right jpeg export launch scripts on GitHub

The result should be the correct number of frames saved (frames starts at 00000) and the message “[rosbag-1] process has finished cleanly”

Hit Ctrl + C to exit

frame00000.jpg 640×480

20161006-udacity-self-driving-car-ds1-lc-frame00000
frame00000.jpg extracted from topic /left_camera/image_color

Convert the jpegs to video

Resources:

License: The data referenced in this post is available under the MIT license.  This post is available under CC BY 3.0

Where to next?

Udacity open sources 223GB of driving data

Following on the heels of another self-driving car developer, comma.ai releasing driving data, Udacity open sourced two data sets from their self-driving Lincoln MKZ.

Udacity’s data is over 70 minutes of driving spread over two days from Mountain View, Calif.   You can read more from the TechCrunch article or their Medium post.

The data is available under the MIT License.

We downloaded the first, smaller data set and started exploring the data.

We also have a page tracking available data sets.

License: this post is available under CC BY 3.0

TensorFlow on Windows

TensorFlow is an open source (Apache 2.0) software library for Machine Intelligence created by Google

Windows 7

The two options are:

  • Run in Docker
  • Run in a Linux Virtual Machine (VM)

Windows 10

With the introduction of the Windows Subsystem for Linux (WSL) Windows 10 users have an additional option:

  • Run on Win 10
  • Run in Docker
  • Run in a Linux VM

 

2014 Top 3 open source alternatives to Google Analytics

Mirrored from OpenSource.com under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license

Top 3 open source alternatives to Google Analytics

Posted October 23, 2014 by Scott Nesbitt

If you have a website or an online business, collecting data on where your visitors or customers come from, where they land on your site, and where they leave is vital. Why? Having that information can help you better target your products and services, and beef up the pages that are turning people away.

The way to gather that kind of information is with a web analytics tool.

Many people and businesses (of all sizes) turn to Google Analytics. But if you want to keep control of your data, then you’ll want a tool that you have control over. You don’t get that from Google Analytics, and luckily Google Analytics isn’t the only game on the web.

Let’s take a look at three open source alternatives to Google Analytics.

Piwik

Let’s start off by taking a look at the open source application that rivals Google Analytics for functions: Piwik. Piwik does most of what Google Analytics does, and chances are it packs the features that you need.

Those features include metrics on the number of visitors hitting your site, data on where they come from (both on the web and geographically), from what pages they leave your site, and the ability to track search engine referrals. Piwik also has a number of reports and you can customize the dashboard to view the metrics that you want to see.

To make your life easier, Piwik integrates with over 65 content management, ecommerce, and online forum systems like WordPress, Magneto, Joomla!, and vBulletin using plugins. With anything else, you just need to add a tracking code to a page on your site.

A number of web hosting firms offer Piwik as part of their one-click install packages. You can test drive Piwik or use a hosted version.

Open Web Analytics

If there’s a close second to Piwik in the open source web analytics stakes, it’s Open Web Analytics. In fact, it has a number of key features that either rival Google Analytics or leave it in the dust.

In addition to the usual raft of analytics and reporting functions, Open Web Analytics tracks where on a page and what elements on a page visitors click, provides heat maps that show where on a page visitors interact the most, and even does ecommerce tracking.

Open Web Analytics has a WordPress plugin and can integrate with MediaWiki using a plugin. Or you can add a snippet of Javascript or PHP code to your web pages to enable tracking.

Before you download the Open Web Analytics package, you can give the demo a try to see it it’s right for you.

eAnalytics

eAnalytics isn’t as well known as the other two tools this article looks at, but that doesn’t mean you should ignore it. It’s a good option if your site gets anywhere up to five million views a month.

eAnalytics shares a number of features with Piwik and Open Web Analytics, including a comprehensive reporting system, a dashboard that you can easily customize, and the ability to track metrics and statistics in real time.

This tool also lets you track one or more Twitter and Google AdWords accounts, as well as track metrics across more than one domain or server. You can also add a button or link that lets visitors opt out of being tracked. It’s a nice touch, and one that people concerned with privacy will appreciate.

Before you download eAnalytics, remember to check the system requirements for your server. That can prevent a headache or two.