How to filter a usenet group using a web distributed newsfeed and a desktop newsreader using SLRN style score filtering using sci.astro.amateur as an example

K. Fisher Rev. 3-19-2008C Orig. 3-11-2008

Goto: Sporge attack list results


SLRN style score filtering is an effective means to return sci.astro.amateur from the sewer and back into an enjoyable daily read.

The overrunning of usenet news groups by net trolls, right-wing facist bloggers and persons with mental health problems has reached the point where most unmoderated usenet newsgroups have become unuseable. Another development in the internet over the last two years is, for urban areas, de facto free universal wireless internet access. A consequence of these two forces is that unfiltered web-based browsing through portals like has become unuseable due to trollers and persons with mental health problems.

The primary tool needed to defeat trollers in this modern Usenet environment are web based usenet feeds coupled with desktop newsreaders that support SLRN style score filtering, short for "s-lang read news".

This note explains how to subscribe to an inexpensive (<$5.00 USD) web-based usenet feed, how to implement a SLRN style filter file - also called a "scoring" file - and how to use one popular freeware SLRN style scoring filter newsreader - XNews.

A prior 2006 webpage discussed using Microsoft Outlook or Outlook Express to filter a usenet newsgroup using a postive filter technique. That solution proved inadequate. I concluded that Microsoft Outlook and Express do not include necessary features to effectively filter a modern usenet newsgroup. In 2008, I switched to a SLRN style scoring filter newsreader using a web-based newsfeed. With these tools, sci.astro.amateur has again become useable. This note explains the process of transitioning from Microsoft Outlook to a SLRN-style-filter, web-based-feed newsreader.

For those of you who remain hesitant to transit away from Microsoft Outlook Express or Outlook for newsreading, it is hoped that these notes will provide a roadmap to guide you through the process. Since transitioning to a SLRN style scoring filter newsreader in early February, the usenet group sci.astro.amateur again has become an enjoyable read.

How effective is filtering by SLRN style filter scoring?

A sporge attack on the sci.astro.amateur newsgroup on Feb. 4, 2008 included 91 posts.

Click each figure to see a higher resolution screen shot -

LST example

Fig. 1 - Raw unfiltered usenet feed on 2-4-2008

LST example

Fig. 2 - Microsoft Outlook Express filtering of the sporge attack

LST example

Fig. 3 - SLRN style score filtering results for the sporge attack

I have used the SLRN style filtering techniques shown below for one month (Feb. 4 to Mar. 4, 2008) on the sci.astro.amateur usenet group. Over that month, about 5 message of by well-known group trollers were not blocked by the SLRN like filter. Most passed messages involve fed-up group regulars losing their tempers and responding to known trolls.

In conclusion, SLRN style filtering is an effective means to return sci.astro.amateur from the sewer and back into an enjoyable daily read.

As a disclaimer, there are some types of sporge attacks which SLRN style filtering cannot defend against. Nonetheless, SLRN style filtering will clean up 95 percent plus of the problem.

Setting up for SLRN style filtering

The major steps in setting up an web-based SLRN style filtered newsreader are:

  1. Subscribe to an online usenet feed serivce.

  2. Local ISP's increasingly do not include full Usenet service as an option. Alternatively, ISPs will offer usenet but limit access to land-line only connections. Increasingly, convienent access to Usenet occurs at internet cafes or while traveling, situations where land-line access to a personal ISP is not possible. For such cases, an internet distributed Usenet feed is the appropriate solution.

    If you do not need roaming access to Usenet newsreading, for example on a roaming laptop, and always access Usenet via your landline home computer, then you may not need to subscribe to one of the following web-based usenet feeds:

    An informal personal January 2008 survey returned the following suggestions for low-cost web-distributed Usenet feeds in ascending order of cost:

    Typically, these web-based Usenet feeds are not permanent archival sites. Tera News, for example, might store one or two months of retrospective messages. Unlike Google, it does not act as a permanent Usenet message repository.

  3. Download and install an SLRN-style-filter enabled newsreader.

  4. An informal January 2008 survey and web search returned the following suggestions for free or shareware newsreader clients that includes SLRN and/or group cross-posting filtering:

  5. Initialize a scoring file (scoring.ini)

  6. SLRN ("s-lang read news") - the original SLRN newreader forms either a backend for SLRN like score filtering newsreaders or whose scoring filter mechanism is replicated in SLRN like newsreaders. At SLRN's heart is a text scoring file - "scoring.ini" in XNews - that applies regular expressions to evaluate each post in a newsfeed. Based on a post's rating, it can be automatically killfiled or highlighted for your attention. A score of "-9999" is the lowest possible score and designates a post for automatic killfiling. A score of "+9999" is the highest possible score.

    Not all SLRN style scoring filter newsreaders support all original SLRN commands. Consult the documentation for the newsreader that you adopt.

Filtering using SLRN style scoring

The SLRN authors' scoring FAQ, the original SLRN 1996 FAQ and the Concordia University sample scoring file provide a good overview of some basic filtering commands. The scoring file for your newsreader usually is edited with a text editor.

You can apply a filter to any standard news field, limited only by your own creativity. Look at the header for any newsposts for ideas. Some of the basic filelds include:

Two types of filtering can be implemented: negative filtering to exclude undesireable content; positive filtering to highlight desirable content. Because you will want negative filtering to take precedence, positive filter commands are grouped at the start of your score file; negative commands are grouped after the positive filter commands.

Negative filtering

Some useful SLRN filtering examples to include in your personal scorefile are:

Positive filtering

Positive filtering can highlight exceptional high-content posters or your own posts:

Again, positive filters should be placed in front of negative filters in your score file.

Filtering name-morphing posters

On the modern Usenet, persons with mental health issues have taken to identity morphing, email name spoofing (a form of identity theft) and the use of anonymous remailers in order to avoid newsgroup filters. Nonetheless, such persons can be filtered. All posters, including myself, have a posting style or a posting "tell" - to use an analogy to poker or business negotiation. A morphing poster's tell can be a particular phrase or uniform markup used in a post.

Specific "tells" for current sci.astro.amateur persons with mental health issues will not be described here. If posted, they will simply adapt their posting pattern. However, anyone of reasonable intelligence can review obvious repetitive troll posts and discover a useable "tell" pattern on which a scorefile negative filter can be based.

Other special SLRN style scoring commands

Depending on how close a newsreader follows the original SLRN software score file syntax, other useful score file commands include:

Using the XNews news client

The remainder of this document discusses how to use one of the specific SLRN style scoring filter newsreaders mentioned above - the freeware XNews reader.

The XNews client has some confusing options. The following user guide focuses on workflow. What does a typical newsreading session look like?

  1. Differences between XNews and Microsoft Outlook Express:
  2. The basic difference between XNews and Micrsoft Outlook Express is that by default action, XNews does not create extensive offline local copies of the entire newsgroup feed. XNews is oriented towards reading messages while connected to the internet. You receive only a current batch of message headers. You read only requested messages selected by double-clicking a message header. There is no massive local "pst" file containing the entire newsfeed.

    As noted above, XNews also does not detect and highlight urls for automatic launching. You have to manual copy links into your browser to read them.

  3. Desktop organization:
  4. The following figure shows a useful organization of mulitple document windows in XNews. A server window is open showing subscribed newsgroups. (Server | click-server-name). A list of available local folders appears in the lower left-hand corner for quick access to sent messages. ( Windows | Open folders). At the lower right-hand corner and along the bottom is the folder of archived locally-stored messages. (Folder | Archive).

    XNews Main Windows

    Fig. 4 - Three of four main XNews Windows

    XNews Main Windows

    Fig. 5 - The fourth read messages window in XNews

  5. Start-up and subscribing to newsgroups:
  6. The three buttons at the bottom of the server window control listing all available newsgroups and subscribing to newsgroups:

    XNews Server Window

    Fig. 6 - Subscribe unsubscribe to newsgroups

    You also need to activate the scoring file - but only the first time that you use the program:

    XNews Server Window

    Fig. 7 - Enable the scoring file

  7. Update the unread message count.:
  8. In the subscribed newsgroup window, right-click on any of your subscribed newsgroups. Use "Get all message counts" to update the counts of new messages in all subscribed newsgroups.

    XNews Server Window

    Fig. 8 - Update the mesage count

  9. Open a newsgroup to read the new messages:
  10. Double-click a subscribed newsgroup to open the Usenet group and read new messages.

    XNews Main Windows

    Fig. 9 - Read messages window in XNews

  11. Archive interesting messages for local storage:
  12. Highlight any message. Press "A" to transfer it to your permanent local storage. Archived messages are shown in the Archive folder at the bottom of the Xnews window.

  13. Reply as needed
  14. Adjust your scoring file to killfile undesireable posters:
  15. You can adjust your scoring file on the fly to killfile undesireable posters on your next read. XNews provides several entry points to edit the scoring file:

    XNews Main Windows

    Fig. 10 - Entry to scoring file from the Server window

    XNews Main Windows

    Fig. 11 - Entry to scoring file from the Message read window

    XNews Main Windows

    Fig. 14 - Editing the scoring file in notepad

    XNews Main Windows

    Fig. 13 - Alternate editing of the scoring file using the Scoring Wizard

    You can killfile specific non-morphing posters by highlighting their sent post and pressing "K".

  16. Once all your newsgroups are read, update the read message count:
  17. Leave at least one Usenet group open. On the menubar, select "Group | Catchup | Catchup and purge". This operation will set the last read message to the highest current message number read and will clear the message buffer of unread messages for this newsgroup.

    XNews Main Windows

    Fig. 14 - Update the last message read counter

  18. To read prior messages, rollback the last message read counter:
  19. Sometimes, you will want to back track and read old messages, for example, from yesterday. To backtrack the message buffer, highlight the name of the subscribed newsgroup, right-click and select "Edit read". Subtract 100 or 200 messages from the message read count as needed. When you reopen the usenet group, the older messages will be displayed.

    XNews Main Windows

    Fig. 15 - Open the Edit Read dialogue from the Server window

    XNews Main Windows

    Fig. 16 - Rollback the last message read count by 169 messages

Sporge Attack Table Results

DatesGroupTotal posts Content SubtotalContent filteredSporge SubtotalSporge filteredSporge passed filter
2008-03-13 to 2008-03-18sci.astro.amateur43026539338310

Values listed are after adjustment of scoring file on initial attack posts.


Hopefully, the foregoing tips will guide you on transition from Microsoft Outlook or Outlook Express to a fully filter functional newsreader. Expect one or two weeks of getting used to your new SLRN style scoring filter newsreader. You are probably reluctant to abandon Microsoft Outlook or Outlook Express for news reading on the grounds that you want to minimize software maintenance. But once you have adapted and Usenet reading again becomes an enjoyable experience, you will be wondering why you did not expend the quarter-day of energy needed to transition years ago.