Software

Up and running with Apache Zeppelin

alex@techbuilders — Mon, 26 Feb 2018 01:08:53 +0000

Up and running with Apache Zeppelin alex@techbuilders Mon, 02/26/2018 - 01:08

Apache Zeppelin is a web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more. It has great features, good support, and the platform is visually appealing and built to scale. The only issue with this solution is the lack of details and assumptions made in the documentation which makes it painful to even get started. We have built a goof-proof accelerator script to get up and running as if you knew what you were doing!

As of the time of this article, Apache Zeppelin is on version ~~0.7.3~~ 0.8.0 and still in beta. The dev team has made some great progress over the past few years introducing Apache Shiro for multi-user authentication and building custom visualization plugins with helium. To get started, the project leads assume you're familiar with, have access to, and want to deploy Apache Zeppelin using docker images or Amazon Web Services. For the guy trying to build a proof of concept (as cheap as possible) and see what it's all about, this is pretty frustrating given you have to have access to and experience with other technologies and platforms. At some point, you end up so side-tracked and strung out from going down rabbit holes that you forget what your initial goal was. My intention is to make getting started as easy as possible for the layman to check out a fantastic open source project. In this tutorial, we'll focus on a single node installation, and scale this out to multiple nodes, incorporating authentication, and installing an SSL certificate.

For this tutorial, I'm installing on a Virtual Machine (Hyper-V specifically) and have had decent performance for small to medium proof of concepts using the minimum requirements below. If you're new to Ubuntu, I suggest working with the Desktop version as it's a little easier to understand if you're coming from the Windows world and like a GUI to work with. I'll be using the Desktop version for clarity sake, the Server version is exactly the same but assumes you're comfortable with the terminal.

Minimum Requirements:

Ubuntu 16.04.03 LTS (Desktop or Server)

4 Virtual Processors

8GB of RAM

25GB of Hard Disk (Ubuntu requirement)

Assumptions

You know how to set up a Virtual Machine (see: https://youtu.be/jsDB3AsCh2k )

Admin access to this Virtual Machine

That's the only assumptions? YES! After you create a bare bones virtual machine, our installer script takes care of (most) of the rest.

Step #1 Find and use the terminal

We pick this up at the desktop of a clean install of Ubuntu 16.04.03. Open the terminal and run

sudo apt-get upgrade

wget https://raw.githubusercontent.com/techbui1ders/apachezeppelin-starterkit/master/latest/install-zeppelin.sh

chmod +x install-zeppelin.sh

sudo yes | ./install-zeppelin.sh

Then go and get some coffee as it's going to take a bit. Behind the scenes, it's downloading and unpacking the following (and it's dependencies) in order

        Java
        Apache Zeppelin (0.8.0 or current)
        Apache Hadoop 2.7.7
        Apache Spark 2.1.0 with Hadoop 2.7
        SparkR interpreter with handy R packages
            devtools
            googleVis
            knitr
            ggplot2
            mplot
            plotly
        Matplotlib & Numpy for Python

At the end of the install, Zeppelin will will have an OK status and a readout of the IP address and port that it's now hosted on.

If you don't see the OK and it failed, check to make sure you have Java installed with

$ java -version

If there is no version listed, install java and run the following to restart Apache Zeppelin

$ sudo apt-get install default-jdk
$ /usr/lib/zeppelin/bin/zeppelin-daemon.sh start

Step #2 Take flight

At this point, the engines are warmed and you're cleared for takeoff. Step outside of your virtual machine and browse to the IP address listed during the install. If you missed the IP address. Check it again using

$ ifconfig

Look for the 'inet addr:' and then use that address plus the port 8080 to get started.

From here, we have one slight adjustment to make in an interpreter to have the tutorials flow smoothly. You're currently signed in as 'anonymous', click the user name in the top right hand corner and a dropdown will appear, click 'interpreter' and search for 'sh'. You'll need to modify the shell.command.timeout.millisecs from 1 minute (60000 milliseconds) to a few minutes (3000000)

Now you're all set to begin! Go into the Zeppelin tutorial and run all paragraphs for the first four tutorials to make sure the interpreters and corresponding libraries are installed and running correctly. The R tutorial is more visually appealing and tends to give you a good basis for what can be done.

The 'Using Mahout' and 'Using Pig for querying data' tutorials don't work out of the box (testing before deployment is hard...mmmk!). We'll spend some time correcting these issues in the install script and will update when completed.

From here, I would get familiar with the interpreters and creating / sharing notebooks. Understand how to edit the variables in the interpreters and how that factors in as you'll need to be comfortable changing this configurations to scale this platform out. Since you've installed Ubuntu Desktop, take some time to visually go to the Zeppelin configuration folder paths (specifically zeppelin.home) to see the files and information contained within them.

Step #3 Go into Orbit

If you don't have an S3 bucket already, set up an S3 bucket and place a CSV into it and connect Zeppelin using your favorite interpreter to do some data manipulation and plotting with it. After this point, you should be pretty comfortable with the basics of a single node installation of Apache Zeppelin installed. Below are some useful links to start the learning process with how to scale this out.

What's Next?

So we've installed Apache Zeppelin on a single node with anonymous authentication. We need to take this from POC to a solution that's a bit more enterprise. Our next articles on Apache Zeppelin will contain tutorials on Apache Shiro for authentication and connecting multiple nodes for more processing power. Hope you've enjoyed! Please comment with any questions.

Thumbnail

Blog Category

Software

Tutorials

Data Science

Apache, Machine Learning, Data Science, Visualization

Blog Display

Thumbnail

Making Sense of Qliks new pricing strategy

alex@techbuilders — Wed, 15 Feb 2017 03:46:56 +0000

Making Sense of Qliks new pricing strategy alex@techbuilders Wed, 02/15/2017 - 03:46

Navigating software licensing can be an incredibly complex, humbling, and frustrating ordeal (I'm looking at you Microsoft) especially when trying to compare across vendors. You don't want to get too 'involved' just to find out they don't have certain features power users need or they are not delivering on their product strategy. While you may receive better support and pricing, going with a smaller vendor carries the risk of holding the vaporware that was once the most innovative product on the market. Thankfully, business intelligence is a very competitive space to operate in and many of the large BI vendors offer similar pricing structures, except for Qlik -until now.

What's Changing for Who?

Consistent with Qlik's long term vision, they continue to treat QlikView as the red-headed stepchild. The pricing changes only apply to Qlik Sense customers, QlikView users will not get to take part in any of the changes and should expect to see business as usual. The main takeaways to the pricing changes are below;

Introduction of the new Professional and Analyzer user model for Qlik Sense® Enterprise, available for both subscription and perpetual licensing.
New packaging for Qlik Sense Enterprise will bundle the capabilities of the following products into the user price for Qlik Sense Enterprise:
- Qlik Qlik GeoAnalytics^® base
- Qlik Web Connectors^®
- Qlik DataMarket^® Essentials
- Test & Dev Servers included (Subscription only)

New pricing for Qlik Sense^® Cloud Business
Qlik GeoAnalytics perpetual and subscription offerings are now the same
Updated discount framework and tiers for perpetual and subscription
Consumption based pricing (coming soon!)

What does it mean?

There are four material changes to understand, one not being so obvious. First, let's tackle the subscription vs perpetual vs consumption piece of the puzzle.

Perpetual is the choice if you plan on staying with Qlik Sense for the foreseeable future and have the budget to take the initial hit. Overtime, the total cost of ownership will be very cheap compared to other major vendors and pricing models.

Subscription is a recurring payment that will result in a total ownership cost being higher than perpetual for long term projects, but ideal for those without the healthy starting budgets. This model spreads the cost over the years as opposed to the large first year obligation. Customers can also scale up and down as their needs change with savings realized earlier than the perpetual method.

Consumption is a 'pay only for what you use' type model which works well if value can be tied directly to consumption. It's a double sided sward that's great when usage is low and value is high (healthcare research?!) and painful when usage is high with little tangible value derived in the immediate term.

The second main change to the pricing is a Professional and Analyzer designation. This means the customer gets to determine who the consumers of the dashboards are and who are the developers or power users. The Analyzer license is generally about 57% cheaper than the Professional license. A user can start out on the cheaper Analyzer license, and then migrate to the Professional license as they become more familiar and needs increase. Also, the Analyzer license inherently applies restrictions as opposed to the administrator applying them, which makes life easy. In a scenario where developers are deploying reports to a lot of non-technical folk who have no need for self-service, the Analyzer license is a great way to save money.

To add more complexity, the third material change is tiered pricing. Tiered pricing means that regardless of designation (Professional/Analyzer), the customer can accrue discounts based on the the number of licenses obtained. For the Subscription based model, the discounts are based on total licenses purchased in aggregate. For Perpetual, the discounts are per transaction.

For example, if the client has 100 licenses and is purchasing an additional 200 perpetual licenses they would get a 0% discount for the first 50, 5% on 51-100, and 10% on 101-250. This equates to a discount around 7% or so over a non-tiered based method.

The fourth and final material change is not so obvious. Qlik no longer requires purchasing licenses in packs of 5. This move towards Professional and Analyzer designation only requires 1 Professional license to begin, and any implementation is going to have a Qlik Sense admin so this plays well for the consumer. Additional licenses can be purchased one at a time, catering to the consumer as needs change - this is a big benefit! One downside detail is that starter packs are no longer available, but all things considered that's a non-issue considering 1 Professional license is still less than the cost of 1 starter pack priced license (woot!)

Great, I'm still confused

In general, your Qlik partner or Sales Rep will be able to help you with your unique situation.

If you're just starting out, I would break the changes down into three steps.

Determine Perpetual, Subscription, or Consumption based on typical user base, long term goals, and funding.
Determine the number of developers you have to get a sense of how many Professional licenses you'll need immediately, then think about number of users you'll need in the testing process and number of power users you'll roll out to first (if different from the UAT testers). You want to purchase just enough to be productive, but not so many they are sitting idle.
Take advantage of Tiered pricing and work with your partner to meet your budget requirements. As a partner, there are a few levers we can pull to make sure the client is a happy camper - utilize your influence and partner relationships.

We have built a Qlik Sense dashboard that should help paint broad strokes from a pricing perspective. Choose the License Model type, number of Professional and Analyzer licenses, the project timeframe and a sample plot of the annual fees will display with a few other key metrics. The pricing used is based on the highest dollar amount per license type, as mentioned in the third bullet there are other levers that can be pulled. Consumption based pricing is not as straight forward, and thus removed from the analysis.

Often times, we have customers interested in the price per user per month. This application is focused on deriving that value and showing the weighted monthly cost per user depending on how long your time horizon is. Generally speaking, the longer you use your Qlik Sense assets, the cheaper it is per user per month.

The enormous benefit of Qlik Sense is that users can have access to an unlimited amount of dashboards without a marginal increase in cost.

An example, using our dashboard for 5 Professional and 100 Analyzer Users, the effective cost in a perpetual model is $26/month after 5 years for access to unlimited dashboards... that.is.AMAZING!

This dashboard can be found on our github page at https://github.com/techbui1ders/QlikSense-LicenseAnalyst . Take a look and let us know if you have any questions or feedback!

Please note this is using USA based pricing as of 2/2018 and is not to be used as a basis of a quote. Please reach out to your partner or rep for an accurate quote.

Keeping your enemies closer

The payment structure and user segmentation was a hurdle that was figured out early by Tableau, and many other vendors followed suit. Qlik has generally taken the position of distancing themselves from competitors by thinking independently. In this case, imitation is just what Qlik needed to keep current customers happy and drive adoption and usage with new customers.

Wrapping it up

Software licensing continues to be complex to the point of giving up, and Qlik is really no different. While annoying to juggle, these changes are a major step forward for Qlik and we look forward to seeing how this new pricing ultimately plays out. The features coming in the 2018 releases should continue to position Qlik Sense as a top player in the business intelligence field at fantastic price points.

Thumbnail

Blog Category

Technology

Software

Qlik, Qlik Sense

Margie (not verified)
Making Sense of Qliks new pricing strategy | TechBuilders

Hey there, You have done a fantastic job. I'll certainly
digg it and personally recommend to my friends. I am sure they'll be benefited from this website.

Tue, 05/01/2018 - 13:34

Stefan (not verified)
Making Sense of Qliks new pricing strategy | TechBuilders

I’m not that much of a online reader to be honest but your blogs really nice, keep it up!
I'll go ahead and bookmark your site to come back in the future.
All the best

Sat, 06/02/2018 - 04:50

Blog Display

Thumbnail

Software

Up and running with Apache Zeppelin

Minimum Requirements:

Assumptions

Step #1 Find and use the terminal

Step #2 Take flight

Step #3 Go into Orbit

What's Next?

Leave a comment

Making Sense of Qliks new pricing strategy

What's Changing for Who?

What does it mean?

Great, I'm still confused

Keeping your enemies closer

Wrapping it up

Leave a comment