Visualization Platforms for Data Search — Amplitude VS Kibana

In the world of Big Data, data visualization tools and technologies are essential to analyze massive amounts of information and make data-driven decisions. Visualization is an increasingly key tool to make sense of the trillions of rows of data generated every day.

Our eyes are drawn to colors and patterns. We can quickly identify red from blue, square from circle. Our culture is visual, including everything from art and advertisements to TV and movies. Data visualization is another form of visual art that grabs our interest and keeps our eyes on the message.

Hence, starting from the data value in today’s business and the data visualization role in making data-driven decisions, in this post, we’re exploring two different visualization tools namely Amplitude and Kibana.

Amplitude

A behavioral analytics platform; helps to track the users’ behavior and gain insights from it to get the complete picture beyond vanity metrics like page views and clicks. It answers complex product questions like, which activities keep users coming back? What is the impact of your latest release? And so on.

How Does It Work?

Every time an action happens that you wish to keep track of, you send an Event to Amplitude. The sent events can be as easy as choosing the text string that defines them. They can also have properties that give you context about them like description, category, type, duration, level, % completed, name, count, source, status, from, number, etc.

Moreover, there are user properties that help understand your users at the time they performed actions within your application. What state they were in, their preferences, their device details, etc.
These properties help you to define different users’ segments.

The activity logs can be queried on the two sides: according to the events that were performed and the users’ segment that performed them. Amplitude has a friendly interface to help you build these queries, and get insights from the users’ behavior by visualizing the returned results in live charts with different types e.g. bars, points, etc. And on different time periods e.g. daily, monthly, weekly, etc.

Available Charts

Amplitude supports several charts that serve the purposes it was created for. We will walk through the most prominent ones in this section.

Event Segmentation

Event Segmentation is the core Amplitude chart that uncovers what users are doing in your product. Some capabilities include:

  • Measuring the top events performed over a selected time period
  • Comparing event totals to each other
  • Understanding which users trigger certain events
  • etc.

Query

Events segmentation provides four default query types:

  • Top Global Events: Queries on the top ten events by volume with the highest counts, over the time period selected, for all users in your project.
  • Top Events for Segment: Queries on the top ten events by volume with the highest counts, over the time period selected, for a particular user segment.
  • Any Active Event: Queries on any active event (e.g. active users), over the time period selected. For example, you want to view your Daily Active Users.
  • Any Event: Queries on any event over the time period selected.
  • New User: Queries on new users, over the time period selected. For example, you want to view your Daily New Users. We define a user as “new” when they send their first event to Amplitude.

Building a custom query is also available:

You can select up to ten events by their names to compare them in one graph, and you can also define the users’ segments you want to study their behavior on these events.
The following clauses help you build such queries:

  • Where Clause: The +where clause conditions the event by an event property or user property. Event property reflects more detailed information about an event at the time it was tracked. User property reflects traits about the individual person using your app.
  • Group By Clause: The +group by clause distributes the event by an event property or user property. You can only group each event by a maximum of two properties.

Funnel Analysis

Funnels help you understand how users are successfully navigating a defined path in your product and where there is a drop-off. A funnel is a series of events that a user progresses through within your app.

Query

We build the query in the same way to the one in Event Segmentation chart, however, here the order of the selected events are taken into consideration.

Retention Analysis

Retention measures how often users are returning to your product after an initial event. So, Retention Analysis displays the average number of users who are new and then return at a later date.

Query

You can select up to two events to chart the average number of users who are new and then return at a later date and complete either the first or second event.

User Composition

The User Composition chart shows the breakdown of active users based on a single user property. 

Query

Select the property you want to segment the users on, you can also define users’ segments.

Stickiness

Stickiness displays the number of days a user performs an event within a week/month. This can be helpful to understand which events are performed by your stickiest users.

Query

You can select one event and determine its properties using +where, you can also define users’ segments.

Custom Visualizations

You can build more customizable visualizations by connecting Amplitude to other tools like Tableau, Sisense, and ChartIO.

Kibana

Kibana data visualization was created to be a window into the Elastic Stack, it is the handiest way to interact with Elasticsearch data and visualize it through dashboards.

Kibana provides countless visualizations and widgets that are designed to improve the reporting experience and help to better understand data. These include area, pie, line, and bar charts, histograms, sunbursts, heat, region, and coordinate maps, gauges, data tables, tag clouds, etc. On top of analytical power, all of the charts are interactive and visually appealing.

How Does It Work?

You should have indexed some data in your ElasticSearch, once you entered the visualize page on Kibana endpoint you can start building visualizations around your data.

There are many features at your service — you can apply metric and bucket aggregations to the axes, select subsets of documents by filtering data interactively based on logical conditions, Kibana provides a straightforward way of filtering the data via the comfy interface, choose modes and colors of charts, and place legends and labels.

To allow end-users to filter the data interactively, you can add control widgets — special elements of the dashboard which allow filtering the data simply by clicking them. Moreover, you can make your graph to be refreshed automatically once in a while and get a real-time picture of your data.

Once you have at least one or two visualizations, you can proceed to combine them. Kibana allows placing multiple visualizations on a single dashboard and reusing them anytime later.

Queries

Queries in Kibana are as simple as the following example:

We perform our example using the Kibana sample data flights, where the used query and the generated chart are shown in the figure above.

Explaining the query:

Metrics aggregation calculates metrics e.g. count, sum, average, etc. over fields.
The “Count” metric used in the example above counts the number of values that are extracted from the aggregated documents.
In this example, the metric is used to count the number of flights.

Bucket aggregations create buckets of documents. Each bucket is associated with a criterion (depending on the aggregation type) which determines whether or not a document “falls” into it. In other words, the buckets effectively define document sets.
In the context of the above example, we used “terms” aggregation where buckets are built – one per unique value for the chosen field.
The chosen field was set to “DestCityName” which is a “keyword” field that denotes the destination of the flight.

Combining these two aggregations we could get a pie chart shows the number of the flights with a specific destination.

Many other aggregation types exist, combining them in a meaningful way allows us to explore our data deeply and get insightful visualizations.

Along with this querying interface, you can choose how to filter the data — either by using the Kibana Query Language (a simplified query syntax) or Lucene.

Custom Visualizations

Due to the open-source nature of Kibana, plenty of developers from all around the world do their best to broaden the available features of Kibana by contributing to its development. Given so, many visualization plugins are available, and it’s easy to build your customized ones.

Embedding Kibana Visualizations in Your App

Kibana allows you to create visualizations using its administrator page, which is a designated UI. you can further assemble multiple visualizations into dashboards.

But what if you want to embed those visualizations and dashboards into your app?

This is also pretty easy with Kibana, it allows you to do it simply by using iframes.

But what if you want to create those visualizations programmatically so they will fit your apps business logic?

Using Kibana’s administrator page, you can only create and update those visualizations and dashboards manually.

Given the fact that the API is exposed as REST URLs, most of the parameters you need to create visualizations and dashboards are stored in the URL state. After creating a sample visualization or dashboard, you should be able to examine the URL and decide what to parameterize.

Let’s say we created a Pie Chart using Kibana, you can create it using Kibana web interface to see the URL:

https://path_to_kibana_endpoint?embed=true&_g=()&_a=(filters:!(),linked:!f,query:(language:kuery,query:''),uiState:(vis:(legendOpen:!f)),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:DistCityName,missingBucket:!f,missingBucketLabel:Missing,order:desc,orderBy:'1',otherBucket:!f,otherBucketLabel:Other,size:10),schema:segment,type:terms)),params:(addLegend:!t,addTooltip:!t,isDonut:!t,labels:(last_level:!t,show:!t,truncate:100,values:!t),legendPosition:right,type:pie),title:'%5BFlights%5D+Airline+Carrier',type:pie))

Which is the same “destination countries” pie chart shown in the previous example.

So, then, we want to create a similar visualization but for the “origin countries” directly, without using Kibana. The URL looks like this — changing only one param which is the “field” param:

https://path_to_kibana_endpoint?embed=true&_g=()&_a=(filters:!(),linked:!f,query:(language:kuery,query:''),uiState:(vis:(legendOpen:!f)),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count),(enabled:!t,id:'2',params:(field:OriginCityName,missingBucket:!f,missingBucketLabel:Missing,order:desc,orderBy:'1',otherBucket:!f,otherBucketLabel:Other,size:10),schema:segment,type:terms)),params:(addLegend:!t,addTooltip:!t,isDonut:!t,labels:(last_level:!t,show:!t,truncate:100,values:!t),legendPosition:right,type:pie),title:'%5BFlights%5D+Airline+Carrier',type:pie))

We can then embed this in our app using an iframe.

Conclusion

In this post, we explored two different visualization tools that complement each other; Amplitude, a behavior analytics platform that helps to get insights from the users’ activities log data, and Kibana, a window into your data stored in ElasticSearch.

We saw how Amplitude is more restricted with lower capabilities in general, however, it’s as simple as it can satisfy the purposes it was created for perfectly with minimal effort. Kibana was shown to be the best-integrated visualization tool with ElasticSearch, an open-source tool with large supporting community, has super customizable abilities, and allows a straightforward embedding way in your own applications.

The power lies in using each tool in the role it was created for, as trying to use it in different context make things complicated, requires more effort from your side to get the desired results, and lead to unclean solutions.

Further Reading

  1. https://dzone.com/articles/kibana-and-beyond-how-to-visualize-elasticsearch-d
  2. https://blog.webiks.com/kibana-api-create-and-edit-awesome-kibana-visualizations-and-dashboards-inside-your-own-app-dynamically/
  3. https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-aggregations-bucket-terms-aggregation.html
  4. https://www.freecodecamp.org/news/powerful-tools-for-elasticsearch-data-visualization-analysis/
  5. https://discuss.elastic.co/t/building-kibana-dashboard-dynamically/33713
  6. https://discuss.elastic.co/t/how-to-create-visualization-on-the-fly-using-a-script/31255
  7. https://discuss.elastic.co/t/how-to-put-a-visualization-directly-in-kibana-elasticsearch/48325
  8. https://amplitude.com/integrations?category=business-intelligence
  9. https://www.elastic.co/guide/en/kibana/master/kuery-query.html
  10. https://discuss.elastic.co/t/embed-dashboard-or-visualization-on-an-external-web-page-with-specified-filters/133961
  11. https://help.amplitude.com/hc/en-us/articles/360033852251-Event-Segmentation-Getting-Started
  12. https://help.amplitude.com/hc/en-us/articles/229951267-Funnel-Analysis
  13. https://help.amplitude.com/hc/en-us/articles/230543327-Retention-Analysis
  14. https://help.amplitude.com/hc/en-us/articles/230901928-Stickiness
  15. https://help.amplitude.com/hc/en-us/articles/231258248-User-Composition
  16. https://www.tableau.com/learn/articles/data-visualization

Leave a Reply

Your email address will not be published. Required fields are marked *