Techie Bytes

Hadooping Around!! :)

Are you in dilemma or as confused as me trying to understand the Hadoop framework and the evolving tool stacks associated with Hadoop? Then try this blog link which explains different processing frameworks of Hadoop and things you need to understand when choosing the framework for your use case - "http://radar.oreilly.com/2015/02/processing-frameworks-for-hadoop.html".

DW Architecture Tech Byte

The Ongoing challenge of Enterprise Analytics/Data warehouse applications is Budget and agility to adapt the evolving sources of data. An excellent article from an IBM architect/Consultant, Explaining the significance of Adaptive Architecture w.r.t Data warehouse applications ( http://ibmdatamag.com/2015/02/what-is-a-data-warehouse-after-all/)

Exploring Tableau Architecture

Familiarizing the BI tool architecture takes you one step further in getting to know better about your BI tools and Capabilities/Offering. Capacity planning for the new implementation still requires basic Hardware and Software experience and Understanding along with the suggestions from the Product Vendor.

In my quest of 2nd week working for Tableau/Reporting support, I started reading this Whitepaper from Tableau - https://www.tableau.com/sites/default/files/whitepapers/whitepaper_tableau-for-the-enterprise_0.pdf.

If you are part of any BI product installation or familiar with architecture, learning Tableau relating to other BI products can be relatively interesting.

Tableau is N-tier architecture with exciting in-memory capability.

Following are the layers of Tableau Architecture.

  • Customer Data - Heterogeneous source systems
  • Data Connectors - Fast Data Engine/Native Connectors
  • Main Components - Data/VizQL/Application
  • Gateway - Gateway/Load Balancer
  • Clients - Desktop/Mobile/Web

Unique offering of Data Layer - I feel Business and IT organizations of the enterprises worldwide pushed the Analytics Vendors to have this capability of combining the heterogeneous source systems (I know what you are thinking, streamlining business process to make use of traditional DW and BI tools can be hard especially when business users have many more excel files to be analyzed.) Explosion of evolving potential Data sources in the day to day business requires IT and Business to be agile and nimble, Tableau and the likes were meant for such business users/process wherein you are requested to integrate many of these data sources with your existing Data warehouse/Decision support systems. Unique offering with Tableau data layer is it can leverage the power of Database engine for the analysis and also on the other hand support the power users with the in-memory Data engine which improves the speed of the data analysis to be in-par with the expectation.

Data Connectors - I like the mode of connection capability with respect to the data source connectivity, Provides ability to report on live data and also in-memory mode to leverage the in-memory data engine.

Especially the below point seems to be interesting, Would like to experiment with some large data extracts. "Because the Data Engine can access disk storage as well as RAM and cache memory, it is not limited by the amount of memory on a system. There is no requirement that an entire data set be loaded into memory to achieve its performance goals."

Tableau Server Components - Tableau server components includes 4 processes.

Application Server (wgserver.exe) - handles browsing and permission.

VizQL Server (vizqlserver.exe) - Handles the data query to Data source which is rendered in desired format. Includes a cache which can be shared across multiple users.

Data Server - centrally manage and store Tableau data sources. It also maintains metadata from Tableau Desktop, such as calculations, definitions, and groups.

Backgrounder - The backgrounder refreshes scheduled extracts and manages other background tasks.

Gateway/ Load Balancer - Similar to the other BI tools, Gateway handles the end user request and assign to the appropriate process and distributes the load. If you have worked on Cognos installation/environments, this also does the same.

Tableau Client Interfaces - Tableau Server provides interactive dashboards to users via zero-footprint HTML and JavaScript (AJAX) in a web browser, or natively via a mobile app.

Tableau Desktop - rapid-fire authoring environment used to create and publish views, reports and dashboards to Tableau Server. Also can access the published views/data sources on the server.

Happy Learning!

Entering the World of Data Visualization – Part I

After being more part of traditional Business Intelligence and Data warehousing projects, recently I made my first job change embracing my aspiration to enter the world of Big Data & Data Visualization. It definitely helps to such Data Integration/reporting background, when it comes to data visualization projects. With the myriad of data sources available nowadays, it is essential for the reporting/Analytics application Client IT organizations and IT vendors are honing their skills day by day in latest Data visualization tools like Qlikview, Tableau and Logi etc available in the industry and the BI/Reporting products were looking forward to enhancing their solution/architecture flexible enough to plug and play the evolving data sources in the day to day business world to derive meaningful insights enhancing the decision making process.

I have a desire to explore Amazon and Public data sets (Which I will get my hands on soon), a treasure trove of Information which can be utilized to extract meaningful insights to help the business.

To start with my journey, I have followed the below 3 steps which I consider are the baby steps.

1) Understand the tool/technology Vocabulary

Tableau File types

File Name/Type

Description

Workbooks

Tableau workbook files have the .twb file extension and are marked with the workbook icon. Workbooks hold one or more worksheets and dashboards.

Bookmarks

Tableau bookmark files have the .tbm file extension and are marked with the bookmark icon. Bookmarks contain a single worksheet and are an easy way to quickly share your work.

Packaged Workbooks

Tableau packaged workbooks have the .twbx file extension and are marked with the packaged workbook icon. Packaged workbooks contain a workbook along with any supporting local file data sources and background images. This format is the best way to package your work for sharing with others who don’t have access to the data.

Data Extract Files

Tableau data extract files have the .tde file extension and are marked with the extract icon. Extract files are a local copy of a subset or entire data source that you can use to share data, work offline, and improve database performance.

Data Connection Files

Tableau data connection files have the .tds file extension and are marked with the data connection icon. Data connection files are shortcuts for quickly connecting to data sources that you use often. It's a small XML file with various connection info this last statement is not from the manual

For More references, refer these links which helped me to understand.

Tableau file types- purpose, features and limitations – https://www.freakalytics.com/blog/2011/08/14/tableau-file-types-purpose-features-and-limitations/

Tableau Community thread on file types – http://community.tableau.com/thread/125511

The Building Tableau Vocabulary Hubhttp://community.tableau.com/message/288134#288134

Visual Tableau Dictionary – http://community.tableau.com/ideas/1851

2) Understanding the discrete and Continuous – Behavior of the tool/technology

It is key to understand the data as well as reporting tool/technology to make the reporting applications as success. Now it is time to wear the hat of DW & BI professional (Data warehousing and Business Intelligence), since this has to do with measures and dimensions.

This excellent article titled “Why are some things in Tableau desktop BLUE and other things GREEN?" explains how Tableau treats the data when you place them in different parts of the report layout (Filters, Report column, row etc).

Reference Link - http://www.theinformationlab.co.uk/2011/09/23/blue-things-and-green-things

3) Art of Visualization

No Matter how great your Analytics is, if you didn’t present the data pleasantly in dashboards the effort put together in terms of data analysis will go down.

Equip yourself and fill your quip with mighty tools and arrows (tips and tricks), this will help in creating/applying the visualizations appealing to the business users.

For example this Knowledge Base article explains how to come up with customer color palette and apply them in your data visualizations.

Reference link: http://kb.tableau.com/articles/knowledgebase/creating-custom-color-palettes

Happy Learning!

Finishing 2014 on a Good Note and Welcome 2015!

Started the year 2014 with no specific plans in mind, Which is more dangerous behavioral aspect or I would call it Nomadic mindset. As often afraid that I have fallen prey to the famous saying - "Ideal man's mind is devil's workshop", I have tried few changes which have fared well and working on few changes which believe will fare me this year.

Books & Reading: It's my nature to get obsessed with reading and attracted towards the good looking companion (My Thoughtful Books!). Glad that this persistent habit, leads me in several tough journey to read, learn and explore more and to achieve and contribute back.

A disappointing failure in 2014 is that, I didn't achieve my goal of finishing the atleast 12 books. Taking the onus solely, I decided to move forward and keep raising the bar, and try to succeed with desperate measures this year in 2015.

Not a very bad year, completed few books. Highlight of which was - "Warren Buffett - Making of an American Capitalist" by Roger Lowenstein and "Take Me Home - The Inspiring Stories of 20 Entrepreneurs from Small Town India with Big-Time Dreams" By Rashmi Bansal.

These 2 books aroused interest in Entrepreneurial and Business related Non-Fiction books, And changed my view completely 360degrees.

Realized, Started attacking the great Behavioral problem (Yes. Procrastination!) to correct the habit and the attitude formed in my mind because of this bad habit.

Personal Finance: Another great year has passed, with more opportunities waiting in hand awaiting for my taking. This year, I overcome the mental burden and strenuous worry about home buying. It's unusual for professionals especially in my industry to ever be without a home buying experience for 9 long years. Blessed with elders' wishes and God blessing, I was able to buy (My Family's dream come true) home back in my home town.

Investment and Life Insurance is whole another chapter which I will start concentrating from this year on.

Relationship and Career Growth: Able to transfer my job dependencies and get relieved from my old role successfully during mid of this year. Gives a chance to me to start yet another new journey in GB, Wisconsin. So far It was a tremendous and highly eventful experience for me, Both personally and professionally.

On Relationship front, reduced my anger to a great extent. Working on improving Mindful Breathing, listening and healthy living conditions.

Wishing you all and your family a Happy and Prosperous New Year 2015!






Understanding of ODI - PartI

interesting comparison of Informatica with ODI, Especially for those who are new to ODI.

I think the CDC features of ODI especially when working Oracle ERP sources like Siebel, EBS and OTM etc have a greater advantage compared to Informatica (Guess the CDC component of Informatica is separately licensed).


H Learning - for EDW Professionals

Data is imperative to any business, Which is why we are in business today. The question now for us is How can we keep up with traditional Enterprise Information management landscape and equip ourselves personally with the evolving Data science and Big Data technology stack. Often the expectation from our customers in this competitive industry, is to be on top of the technological evolution and keep abridged with our latest industry trends. This factor also helps Organizations to form CoEs, forums etc which will contribute in developing the niche skills and capabilities, which will indirectly benefit our solutions we provide to existing customers and to win new businesses/deals.

This factor and expectation drives the interest to learn more about these latest technological trends and potential impact on traditional DW environments. I have been following passively by reading some blogs, forums etc to boost my understanding about the latest Big Data/Hadoop trends.

You will find below my favorite links and blogs I liked while exploring. I intend to continue on this journey as and when I found interesting piece of wisdom/information.

Hadoop in 5 minutes: A inspiring blog by MapR CEO explaining the top use cases of Hadoop (Like to Mention first use case is Aadhaar project - adhaar is providing a unique identifier for every resident of India, so that's 1.2 billion residents.).

https://www.mapr.com/blog/hadoop-5-minutes-or-less#.U_d3XJRdXrw

Hadoop 101 for EDW Professionals : Ralph Kimball explains how Hadoop can be both a destination data warehouse, and also an efficient staging and ETL source for an existing data warehouse.

http://www.cloudera.com/content/cloudera/en/resources/library/recordedwebinar/building-a-hadoop-data-warehouse--hadoop-101-for-enterprise-data-slides.html

Keep Abreast..

Is it too much information nowadays to keep abreast in industry? Just the mere thought/desire of following interesting books, reading related discussions in sites like goodreads, Pinterest, FB, quora etc seems daunting. Also more challenging task is to record your thoughts/actions occasionally to review your learning/activities. Where I am trying to strike a balance, Determining the amount of information which is helpful, avoiding bombardment of information from too many sources.

It is good to keep updating yourselves about the industry news and updates. Gives a little of confidence when we talk with the industry peers and superiors in Organization to demonstrate our understanding about the industry.

The recent release of Qlik Sense product by Qlik caused ripples in online discussion forums. Usually experts compare Qlik with the arch rival Tableau, When it comes to evaluating Qlikview.

I was doing some reading in the below link about Qlik Sense FAQs in my quest to understand how Qlik sense is different from Qlik view (understand both are different products from the Qlik tech).

Qlik Sense Faq

Here are some of the questions I have on Qlik Sense:

What is Qlik Index Platform (QIX)?
What is Qlik Sense and How is it different from Qlikview?
What is Qlik.Next? (A strategic program targetted @ improvements to Qlikview Capabilities?)
Similar to VizQL in Tableau, does Qlikview or Qlik sense uses similar method to combine data from heterogeneous information?

Can I run Qlik Sense Desktop in a web browser?
Yes, once Qlik Sense Desktop is started point your supported web browser to http://localhost:4848/hub/

Seeking experts help for some of my questions above.
Good to know that Qlik Sense is free and unlicensed Product, interested in trying this product in some spare time.

Happy learning!

Cinephilia - "Ferris Bueller's Day off" movie review

Awesome! I was searching and I didn't get a much better word to describe the movie. Brilliant and engrossing entertainment.. Yes, I was talking about the movie "Ferris Bueller's Day off".

It brings back the memory of younger days and lots of dilemma we all faced during our school days. Faking the sickness to avoid school attendance - A classic, remarkable and evergreen.

It's a John Hughes film shooted in chicago downtown back in 1985/1986, showcasing famous attractions like Sears tower, Art institute of Chicago, Skyline and Wrigley field. Wikipedia claims that this film according to Hughes was dedicated to his love on Chicago and he calls it "A love letter to Chicago".

Being encountered ostentatious Chicago before, I could relate some of the places even today after almost 3 decades. City's downtown and Astonishing skyline always treat to One's eyes.

Avoiding further spoiler alerts, and still interested/curious to explore/learn about the film's musical tracks which is another good feature adding feathers to its laurel.

Also thanks to one of the Academy trainer (forgot his name) who suggested/mentioned this movie during one of his training. That triggered a interest in me to watch and explore.

"The World is a book and those who do not travel read only one page" - Read Online.

Wintered!!

Because of yucky outdoor weather (Which is sooo cold!), I happened to start watching this series called "Californification" (Intrigued by the title which is similar to the following word I come across - Bengaloored, the meaning of which I forgot). It's not related to Silicon valley which is a software hub in the state of California. Not very much interesting to watch with family (I am speaking in terms of traditional Indian family values, for those western culture is shocking revealation)
Caution: Meant for Adults :P

Book Review - "Buffett: The Making Of An American Capitalist"

Where do you want to be in 5yrs, 10 yrs? This is typical question we ask ourselves over some time to streamline our goals and progress in life. But can you be believe, it's been 50years since Mr. Buffett has started guiding his crewmen in the Berkshire Hathaway. When I come across about this fact in the news few months back, I was intrigued and wanted to know more about the “Sage of Omaha" and his juggernaut Berkshire Hathaway still continuing to advance/strive forward.

This interest in me, made me to look for books about him in Bookshelves. Of course there are lots of books about him in the market, but happened to get this second hand book for a cheap price in a local store. After buying this book, I kept it without reading for a long time. This might be because of my surreal mind set suffering from fiction infestation from the likes of John Grisham, Suzanne Collins, Veronica Roth etc.

Since the Author is a finance columnist he has the tendency to follow Mr.Market and Mr. Buffett by nature, which helped him to relate and unearth the hidden jewels about the Oracle.

Without any further spoilers, I would highly recommend this book to anyone those who want to know more about Buffett’s lifestyle, interests and background. You get to know about his unique method of adapting “Value Investing based on Intrinsic Value" from his famous teacher Benjamin Graham. I was introduced to Benjamin Graham and his famous book “The Intelligent Investor" (Which is still in my “Want to Read" bookshelf) by my friend few years book.

Being from Non-Financial/Economic background, often at times I was fighting and dragging me skimming few stanzas without trying much to understand the context which I hope might be fixed during my re-reading plan.

Being from India, I was captivated by the name “Ajith Jain" an only Indian Origin employee of Berkshire Hathaway ever mentioned in this book, who happened to introduce the Super-Cats Insurance. Well you might ask, what is Super-Cats Insurance? I suggest why don’t you look up online.. J