The Essential Guide to Content Moderation
Content moderation is the act of applying a set of guidelines to text, images, and video that appear on a website, often with a particular focus on user submissions. It involves monitoring and identifying potentially harmful content, assessing whether it complies with the site’s guidelines, and filtering out anything inappropriate.
Moderating content is a complex task, often involving many additional processes that go far beyond the controversial work of vetting violent or extremist content. In this article, we’ll talk about what content moderation involves and introduce a range of different ways that content is processed and filtered. After all, despite the many challenges facing the industry, content moderation is becoming an essential part of online business. A clear understanding of it can help you to build a closer relationship with your customers – and protect your brand’s reputation.
This article was written by Daniel Smith and originally published on Lionbridge AI
What constitutes sensitive content?
The main purpose of content moderation is to remove inappropriate content from a website. This often comes in the form of graphic or extreme content which contains violence, hate speech or nudity. Depending on each site’s specific requirements, moderation can be undertaken to a greater or lesser extent. Some message boards pride themselves on freedom of speech, while others, such as social networking sites, have to strike a difficult balance between ease of use and protecting their younger users.
However, there are also a range of UX challenges to keep in mind when moderating content. Users have come to expect consistently high standards across the websites they frequent, which also stretches to user submissions. As such, it’s important to quickly identify and remove duplicate content through a process called deduplication. Similarly, low-quality image or video submissions are also a frequent target of moderation efforts.
A large number of sites also have legal issues for their moderators to consider. For example, libelous or copyrighted content can cost a company thousands of dollars in legal reparations. It’s a matter of urgency for many businesses to identify and take down this content before it’s flagged by any external party.
Which content types can you moderate?
In addition to the many different ways that content can be sensitive, there are also several different types of content that moderation services have to contend with. Content moderators or algorithms will usually have to deal with at least one of the following content types:
Text
The sheer variety of text that requires moderation on a site can be staggering. From comments and forum threads to full-length articles hosted on your site, almost any type of text can require assessment. As such, moderators and moderation algorithms must be adept at scanning texts of varying lengths and styles for unwanted content.
Furthermore, text moderation can be an extremely difficult task due to the complex nature of language. In order to detect cyberbullying or hate speech, for example, it’s necessary to move beyond explicit keywords and look at whether phrases, sentences, or even paragraphs as a whole breach your community code of conduct. While text may not contain any obvious indicators of sensitive content, it may still contain behavior that is extremely damaging to your site.
Image
Although it might seem simple to identify inappropriate images, there are many challenges to consider when moderating them. For starters, detecting nudity or explicit imagery in user-submitted content can sometimes depend on context. What constitutes an indecent image in the US is very different to an indecent image in Saudi Arabia. Some companies may also draw the line in different places depending on their product, such as lingerie brands, where a certain amount of nudity is required to market and discuss their offering. As a result, providers of image moderation services also have to consider the target audience, market, and even company in question as they monitor a site.
Video
Graphic or violent video content has been the root cause of many of the controversies surrounding content moderation. This could be due to the fact that it’s also one of the most difficult types of content to moderate. While images and text can often be vetted quickly, video can be extremely time-consuming, forcing moderators to watch a video all the way to the end. If only a few frames of video are explicit, it could drastically change the viewer’s perception of the site that hosted it. If your platform allows video submissions, searching for these hidden breaches of community guidelines can have a significant impact on your moderation efforts.
Video moderators are also required to perform several tasks simultaneously. In addition to the actual video content, any attached audio or subtitles need to be vetted for explicit language or hate speech. Separate from the actual video content, audio or subtitles may not match the video accurately or meet the necessary quality thresholds. On top of many of the market, audience, and quality concerns outlined above, this makes video moderation a formidable challenge.
Types of Content Moderation
The content moderation method that makes the most sense for you will depend on your website’s goals. It’s important to consider whether you want people to be able to communicate quickly and easily, or whether it’s more important to keep your site completely free of sensitive content at all times. There are a range of different types of content moderation which fall at varying points on the spectrum between these two goals. The most common ones are as follows:
Pre-moderation
Unsurprisingly, this involves all user submissions being placed in a queue for moderation before they are displayed on the site. Through pre-moderation, it’s possible to keep all sensitive content off a site by checking every single comment, image, or video. However, for online communities that prize immediacy and barrier-free engagement, this moderation method can cause significant problems. It’s best suited to sites which need high levels of protection, such as those frequented by children.
Post-moderation
In cases where user engagement is important but a comprehensive moderation program is still required, post-moderation is often a good choice. It allows users to publish their submissions immediately, but also adds it to a queue for moderation. This allows both immediacy and for moderators to monitor behavior. However, since every comment is approved by a moderator, scalability can be an issue. Since all content must be approved, the site may also be liable if anything inappropriate slips through the net.
Reactive Moderation
For a scalable programme that relies on community members, reactive moderation is a possible solution. This type of moderation asks users to flag any content that they find offensive or that breaches community guidelines. By involving users in the process, reactive moderation directs moderator efforts towards the content that most needs their attention. However, there’s also the risk that offensive content will remain on site for long periods of time, which could damage the reputation of the site.
Supervisor Moderation
Similar to reactive moderation, supervisor moderation involves selecting a group of moderators from the online community. Also known as unilateral moderation, this system gives certain users special privileges to edit or delete submissions as they use the site. If supervisors are selected carefully, this method can promptly remove sensitive content and is easily scaled as the community grows. However, it is also prone to the negative effects outlined above if moderators miss offensive text, images, or video.
Commercial Content Moderation (CCM)
CCM mainly involves monitoring content for social media platforms. It is often outsourced to specialists, who are tasked with ensuring that the content on a platform abides by community guidelines, user agreements, and legal frameworks for that particular site and market. Since the work is performed by specialists, a good standard of moderation is usually guaranteed, despite sometimes difficult or controversial conditions for the moderators involved.
Distributed Moderation
As one of the most hands-off moderation systems, distributed moderation places a lot of trust and control in the hands of the community. It usually involves allowing users to rate or vote on submissions that they see, flagging content which goes against any guidelines that are in place. This often takes place under the guidance of experienced moderators and can work well if a site has a large and active community. Despite this, distributed moderation systems remain somewhat rare. The risk of allowing a community to almost entirely self-moderate is one that many companies aren’t willing to take, particularly because of the negative impact this could have on their brand and reputation.
Automated Moderation
Finally, automated moderation is an increasingly popular moderation method. As the name suggests, it involves the use of a variety of tools to filter, flag and reject user submissions. These tools can range from simple filters, which search for banned words or block certain IP addresses, to machine learning algorithms, which detect inappropriate content in images and video. For now, many of these tools are used in addition to some kind of human moderation, but as they grow more sophisticated in their ability to analyze conversation they may become a viable standalone option in the near future.
Who performs content moderation?
The difficult working conditions endured by human moderators have been well highlighted in the international media. The stressful nature of the job, which involves constant exposure to some of the most extreme content on the Internet, has led to an extremely high turnover rate for these positions. As content moderation’s massive impact on the mental and physical health of moderators is further exposed, the use of machine learning to take on the bulk of this work looks like an increasingly desirable option.
Machine learning algorithms are already being implemented in a variety of ways to remove the burden of extreme content from human workers. These algorithms are built on the work of their human predecessors, since they are trained using large datasets of previously tagged content. From this huge bank of relevant examples, the algorithm extrapolates the rules that govern the distinction between safe and sensitive content. An intricate understanding of these rules allows algorithms to flag explicit material with increasing accuracy.
The next step in the development of these models involves building out the capabilities to deal with some of the more complicated instances of inappropriate content mentioned above, such as cyberbullying. Future content moderation tools will be able to calculate a relative ‘risk’ score for a piece of content, before determining when and if it should be reviewed.
For now, the sheer complexity of content moderation means that full automation remains a dream from the distant future. However, as the industry adapts to protect its workers, human-in-the-loop moderation workflows will become far more common. These build on the strengths of both human and machine, allowing the algorithm to square away a large proportion of inappropriate content while referring difficult, subjective content to their human overseers. In this way, the continuing expansion of AI in this field will help to protect not only Internet users, but also moderators from the worst of the world of content.
Why is content moderation important?
It’s absolutely crucial to build a solid and lasting connection with your customers. Creating an online presence where people can engage with you will not only expand your customer base, but also give you valuable insight into how to improve your product.
However, these strategies necessarily come with risks attached. By opening your business up and creating a community, you also open the doors to inappropriate content from some of the darker sides of the Internet. It’s easy to argue that this only affects companies with certain products, but in reality it affects all businesses. It only takes one piece of inappropriate content to destroy a relationship with a potential customer – after which it’s difficult to win them back.
By using content moderation to enforce high standards upon your site, you protect yourself from a variety of legal issues, protect your growing community, and ultimately show that you care – not just about the environment you create, but about the way that your company engages with the world.
For info on what Lionbridge can do to safeguard your users from unsuitable content, take a look at our content moderation services.
This article was written by Daniel Smith and originally published on: https://lionbridge.ai/articles/the-essential-guide-to-content-moderation/