How to Visualize a Political Bias Metric

How to Visualize a Political Bias Data Metric

One of the services we provide at Almeta is estimating the political bias of a piece of the news in other pieces we have discussed the technical details of this feature but in this piece, we will go through the possible suggestions to visualize this calculation, with the main goal of facilitating the process of taking feedback from the users and retraining our models using it.

Why you should be Interested in this piece?

Well if you are an ML guru or a newbie your goal would be to convert your model into a real world service, if so then this piece might give you ideas on how to present your million dollar idea.

This article is a part of our series on political bias detection we will hopefully introduce you to the various aspects of our political bias detection system, and you can learn about:

Notes

  • The following proposals are based only on 2 factors, first having a clean design, and secondly the ability to incorporate user feedback in a simple way.
  • we are assuming that the user feedback would be towards sentiment analysis (or subjectivity) and not target detection, as it would be too comprosome on the user side.
  • The agree/disagree markers should be specialized according to the task. e.g. for the overall bias metric the tags should be instead (too high/ too low). This should be designed to incorporate usable and meaningful users feedback

Zoomable timeline (or histogram or Numbers line)

What is it

In the current design we have a spectrum stretching from negative to positive, it would be awesome to embed the targets and their polarity in it. Here is an example, it is really similar to this. the timeline should stretch from -1 to +1, it is zoomable and therefore the user can decide to focus on a certain part of it (negative or positive) and the targets are represented as events which when clicked would change the main layout. In this main layout we should put the target inside the sentence and the user has the option to agree or disagree with sentiment tag.

Other options include using a zoomable histogram something like brushable-histogram here is a demo, the height of each block can be used to represent the target confidence, while the relative location on the spectrum can be used to represent the sentiment confidence. Again when a certain block is clicked it should display the target and the sentence in which it appeared alongside the agree disagree button. Alternatively we can replace the details layout (the one containing the sentence and the agree disagree) with a small pop up containing the sentence and tiny icons for agreement and disagreement.

Finally in both cases we can use the details layout background color to signify the sentiment towards the target.

Pros

  • Simple design
  • tightly integrated many information cramped in a single chart
  • has a scientific sense to it

Cons

  • Users might not be to inclined to do feedback
  • it might be hard on the users to understand the chart especially in the case of zoomable histograms and would need a lot of visual aid for the users to maneuver them
  • seems more suitable for a dashboard
  • hard to embed target confidence in the design (especially in case of timeline) the histogram can indicate that by setting the block height to match the target confidence and by setting simple markers on the X and Y axis (e.g. X title could be ‘How much emotions’ and Y title could be ‘how much I am sure of it’)

Clickable text highlighting

This goes along with the fact that we will display the text in our site, the design entails having several lenses that can highlight the text in different ways this can include (subjectivity, ELSA, errors, fact-checking, …). for example for the case of ELSA the API will return the text and the locations of targets as word indexes. the user would normally see the plain text without any annotations and then after clicking a button the phrases get colored based on the degree of bias from yellow to blue and we can visualize the target confidence using the alpha value of the color. We should as well add a legend to mark the meaning of the words.

Furthermore, we can use the sentences that contain targets to represent sentences with emotion in the text, therefore, creating a second lens.

Finally, for user feedback, we can attach small agree/disagree icons to each of the highlighted segments of the text that appears on hover, this can allow for specialized feedback (especially for emotional sentence lens feedback)

A simpler way to do highlighting in some options like summarization or keyphrase detection is to simply fade the rest of the text instead of highlighting the needed part, this can be helpful in resolving the issue of complicated legends

The user feedback of the highlight can be added using chips in a similar design to the following image but instead of modifying the content it merely adds the agree/disagree icon, see the following diagram

Pros

  • Can be reused by other models including stuff like summarization as we can highlight the summary only and fade the rest of the text
  • can allow for simpler users feedback than the first option yet I am sceptical

Cons

  • The multiple lenses and multiple Legends can be confusing.
  • Users are not inclined to do detailed feedback

Clickable back2back histograms

This is really similar to the first option however instead of a single histogram representing the distribution across sentiment we have 2 histograms each representing a class (positive or negative) in a similar manner to this image.

Each of the blocks will represent a single target, the height of the box represents the sentiment confidence of the target and the blocks are ordered either based on their target confidence or based on their location in the text (I prefer the former option).

The colouring of the blocks will also be based on the confidences in a similar manner to the highlighting option. Basically the sentiment score will decide the colour while the target confidence is portrayed using the alpha value.

To enable User feedback hover over the boxes displays the target within the sentence and clicking on it enables the agree-disagree option.

This might be a useful tool (a starting point).

Pros

  • Simple design
  • tightly integrated many information cramped in a single chart
  • has a scientific sense to it
  • I feel it is easily understandable in contrast with the first option
  • target confidence is displayed in a straight forward way

Cons

  • Users might not be too inclined to provide such detailed feedback
  • seems more suitable for a dashboard

Conclusion

In this colorful piece, we have went through the process of or UI design and how you should think while choosing a cool interface about the way to harness user feed back and use it to improve your service.

Do you know that we use all this and other AI technologies in our app? Look at what you’re reading now applied in action. Try our Almeta News app. You can download it from google play: https://play.google.com/store/apps/details?id=io.almeta.almetanewsapp&hl=ar_AR

Leave a Reply

Your email address will not be published. Required fields are marked *