[FORUMS] A call for help: spambots

Hi!

Do you have any experience with forum administration? If so, perhaps you can help us.

It's easy to miss it, but our user list is riddled with spammers. They often do not post, but still have advertising in their signature or biography profile fields. Common patterns include:

• A meaningless user name or user title (the line of text under the user name)
  
• A birthday date sometimes set to the day after registration, probably to lengthen the time the user profile stays on the front page (first as "latest user", then in the birthday list) and increase SEO
  

I sometimes check the latest registered users and have a look at their profiles. Most of the spammers have obvious commercial links and get banned immediately (I also clean the profile). Some are more ambiguous and require checks on the IP, registration email or user name to make. I'm sure some of them slip through my checks, and I don't have time to check everyone.

Currently we have an Akismet plugin (which is mostly useless because this acts on posts, not on profiles), as well as a random security question which requires a tiny bit of reflexion and time to solve on registration. This isn't very useful at all, but I guess it's still better than nothing.

Since most of those spammers don't post and we spot those that do most of the time, perhaps we can restrict checking on users without any posts. One suggested idea is to prevent profile edition until users make one post, and pruning accounts with zero posts and one year of inactivity to get rid of the inevitable undetectable spam accounts that would ensue. That way, the only way spam can get through is if it's in the user name, or if the spammer makes a post, which is more easily detected. A few downsides: it's not a permanent solution, as spammers get creative; and it causes minor annoyance and confusion for new users that want to edit their profile. Also, it would need a custom forum modification.

We could also have forum database queries to assist admins in finding suspicious accounts, but those still need to be scanned manually. It's more straightforward to implement and maintain, though.

In any case, we will have to get a DNSBL or other spammer database-based blocking system on registration. Those systems aren't perfect, but they are sure to help.

I don't know how to close this post. Does anyone have anything to share about their experiences or knowledge with spambots? Thanks in advance!

Do you have any experience with forum administration? If so, perhaps you can help us.

Many years ago I was asssociated with a forum that had terrible spam. It got to the point that I set up a phpBB forum specifically to test out anti-spam addons. It worked fine for 6 months until the spambots advanced to a point that they got past the additional checks. I ended up shutting it down.

I sometimes check the latest registered users and have a look at their profiles. Most of the spammers have obvious commercial links and get banned immediately (I also clean the profile). Some are more ambiguous and require checks on the IP, registration email or user name to make.

Some of them are now becoming quite clever with social engineering. I believe many of these are not true bots and actually have a real person doing some of the work, they just have a set of extensions in their web browser and a set of standardised pre-composed posts to allow them to get things posted quickly. Only later do they drop the spam payload into existing posts by which point they may not even be read by a real person, hence functioning only for SEO. These ones are always going to get through until someone notices that it's too late.

as well as a random security question which requires a tiny bit of reflexion and time to solve on registration. This isn't very useful at all, but I guess it's still better than nothing.

Every single thing that adds to their workload (either human or computer) makes their operation less commercially viable and they're only in it for the money. How much of an IQ test do you want to make the registration process? Would it really put off genuine posters to be asked a few further questions?

There are some extreme measures that I can think of but I'm sure these are best avoided:

- Charge a registration fee. It might sound mad but even if you made it a token €0.01 charge that had to go through before you can post it would be impossible for them to ply their trade. Put in the wrong payment details and they wouldn't be able to post, do hundreds of dummy registrations and the amount would add up massively. Would it put off genuine people though? A real concern and I don't know how people would take it. Xonotic is a non-profit group and you could say that it was to fund development and people could voluntarily donate more or you could refund people their €0.01 after x number of posts should they wish to not donate.

- What's already in use by Paypal and others and that is that you have to provide a phone number and an SMS is sent with an activation code that has to be entered. This greatly adds to the effort required for a registration if you're working in an Elbonian spam factory but is little extra work for a real person making just one registration. Would their be a cost for setting up such a system? Probably. There are free SMS sites around but I'm not sure about their own security, if we used them would we be setting up all users to receive SMS spam?

One suggested idea is to prevent profile edition until users make one post

One post is not enough. I'd really be saying at least three posts as this gets you past the fake personal introduction posts that then get edited later. Also add to this that they shouldn't be able to post links, pictures or a signature until this probationary period is complete.

pruning accounts with zero posts and one year of inactivity

One year is too long. A genuine user might sign up and not post for a few days but after a year the chance of such a user even remembering their login details is slim. Make it a few weeks.

Also, it would need a custom forum modification.

A custom modification does have it's advantages. Forum spam software writers will have just as ready access to released forum addons as any forum administrator and they will reverse engineer from them to improve the performance of their software. Making custom changes (I have heard reversing password and username input box names is quiet successful against some bots) confuses spambots and spambot authors won't make changes specific to one forum.

We could also have forum database queries to assist admins in finding suspicious accounts, but those still need to be scanned manually.

Why not use search engine results against them? Google or any other search engine gives you massive amounts of data and it just needs a little clever interpretation. With every first post do an automated Google search for the explicit text posted in the first post. Forum spammers and spam bots aren't very creative and if you find thousands of posts on many forums with the exact same text, they're a spambot. This is the main confirmation I use to decide if someone is spamming.

Other ideas:
- Add an extra set of forum rules into the process. Specifically make it about forum spamming (not for their benefit but for everyone elses), have a set of fake username and password entry boxes hidden from human view and if anything gets entered, don't proceed. This could catch bots only.
- Have a fake CAPTCHA. A jumble of letters with a small input box below labelled 'Don't put these letters in the box below', if the spambot tries to be clever on the CAPTCHA and enters something in the box, kick them out of the process.
- Make the process specifically slower. If you don't take at least 20 seconds on something, you're probably a spammer. This is very useful on forum posts themselves. If someone is copying and pasting then they'll be posting every few seconds so put in a check for how long they take and postpone the process if they take too little time.
- For the registration email change around the format a bit, put in a few links to other parts of the site first, then if a bot or spammer reads the email and just clicks the first link they'll be delayed in registering. This won't make things any worse for real users and possibly better by linking to some useful things like the bug tracker, the dev Wiki, other helpful stuff.

I was hesitant to ask people to post smart and decided it wasn't necessary.

Well, consider it done now.

I've tried DNS blacklisting and worked pretty well (on a low-traffic website at least).

Maybe disabling hyperlinks (in their post, profile and signature) for members with less than so many posts.
This can prevent at least some spam but it won't reuce the number of spammy profiles.
A similar approach is to set nofollow for such links, so that users can follow links posted by legit users but spammers don't get traffic from serch engines.

I was hesitant to ask people to post smart and decided it wasn't necessary.

Well, consider it done now.

I'm not sure if you're referring to my 'blank' message above but there is a message there which I posted and whenever I try to edit it the message is there but what ever I do it won't post. As a moderator maybe if you try to edit it you might see the post which I wrote.

Edit: fixed it now, some funny BBcode issues...

I'm not sure if you're referring to my 'blank' message above but there is a message there which I posted and whenever I try to edit it the message is there but what ever I do it won't post. As a moderator maybe if you try to edit it you might see the post which I wrote.

Edit: fixed it now, some funny BBcode issues...

Ah. The message I replied to was humorous and didn't contribute much at all to the discussion. You fixed that later

Anyway, to address a few of your points: requiring verification by payment is out of question. I don't imagine minors having to ask their parents to pay a small amount of money to register to a forum, I'm sure many of them would think it's a scam. Some adults don't even have access to online payment. SMS auth is more reasonable but, as you say, we can't really provide that reliably.

As for the social engineering side of the issue, you are indeed right. But from my limited experience on this forum, those that have tried have failed. They get more exposure and are more likely to be spotted in time, as opposed to the stealthier profile ads.

I suggested one year of inactivity as a delay because any lurker that does not plan on posting can be expected to have a few weeks of inactivity. Setting the threshold to something around one year is more reasonable in that respect. As long as there is a delay, we can prevent a pileup of spammers. One downside of longer delays is that I have noticed spammers reconnecting some time after registering, even though they were already banned. That means only one connection a year is required to keep an account alive. Someone would have to run some stats on the large amount of tagged spam user accounts we have accumulated over the years.

Custom forum plugins are indeed a good thing to have, but they require initial work then maintenance, and I don't know if anyone in the team is ready to put time into this. We don't want to break the forum database because of incompatibilities.

The message I replied to was humorous and didn't contribute much at all to the discussion. You edited it after.

No, that message would have been my signature and it's still there at the bottom of each post. That's all I saw as well as the message that I posted just appeared blank, trust me. Only when I edited the original message and sorted out some wierd BBcode issue did the proper message appear. Stupid but true.

Anyway, to address a few of your points: requiring verification by payment is out of question. I don't imagine minors having to ask their parents to pay a small amount of money to register to a forum, I'm sure many of them would think it's a scam. Some adults don't even have access to online payment. SMS auth is more reasonable but, as you say, we can't really provide that reliably.

Agree with you on the potential scam idea but this is an absolute extreme method and not a serious suggestion. It has been a suggested before as a method of stopping email spam - just replace the entire system with one based on charges and even if the charges were very small spammers would go out of business.

Perhaps taking the SMS authorisation idea a little differently, what about having an authorisation code or message which is given on one page of the sign up process and has to be entered in on the next page? Or a code that is sent to you in the sign up email and you then have to manually follow a link and type in the code? Or once the account is setup a private message gets sent with an additional link to be clicked?

As for the social engineering side of the issue, you are indeed right. But from my limited experience on this forum, those that have tried have failed.

They are generally unsuccessful and occasionally give us a laugh with their discussion of their other hobbies I agree. If profile spam is the main problem then maybe you need to be really hard on zero posting members? Only allow the accounts to remain active once they post something. Genuine users could be warned of this and it is not uncommon for commercial sites to remove non-active accounts.

Edit: now we're in a preemptive edit war on our own posts. :-)

---

Here's a few zero post accounts picked out with signature spam in them:
http://forums.xonotic.org/member.php?act...e&uid=4594
http://forums.xonotic.org/member.php?act...e&uid=4583
http://forums.xonotic.org/member.php?act...e&uid=4569

I only picked those from the memberlist because their names are suspicious. There's another newer one 'marhar' I have my eye on now as apparently he has only spent 22 seconds online. That sounds like making a registration for the sake of it without doing any search:
http://forums.xonotic.org/member.php?act...e&uid=4607

Is there really much of a reason for people to open an account without posting? A genuine user might do so to search the forums but they could just as easily do a Google search. Is there any reason why the search function is available just to members? If non-members could search then would it make it far easier to distinguish between genuine registrations for the purposes of posting and profile spam?

---

Even better, a user who's been online for 1 second!
http://forums.xonotic.org/member.php?act...e&uid=4575

Yeah, rum jello shots to them too.

If they are real people how about spamming them back? Might sound mad but you could do it:
1. Firstly to cover the forums and do it legally you need to have something in the signup to say that spammers who get banned will receive thousands of emails.
2. Then when you ban someone for spamming, have a tool setup to launch a counterstrike of thousands of emails telling them that they are banned for spamming. If they get caught quickly their inbox will explode and the distuption to their 'business' will be severe. How can they sign up to other forums when there are thousands of other emails coming in? Plus it would be a nice revenge!

I like the idea of micro payments against spam, but it's impractical and I don't want to think about the significant economic consequences as this scales up.

The intricate click and copy and paste schemes you describe are fine ideas, but I don't know if that's any efficient at all.

Thanks for reporting the users. For the record, I just banned them and wiped their profiles. However I believe marhar is legit, after checking email and IP address and finding coherence, so this user won't get banned. The email contains a user name that can be found in a forum consistent with the IP address' origin and email host. It could be well done impersonation but I don't think spammers go through that trouble yet.

I insist that user should be able to lurk without having to ping back every second week. Some people might want to be able to send and receive private messages (if I want to contact someone I found on a server, the first place I would look to contact them would be the forums). Requiring one year of inactivity before blocking then removing the account seems reasonable to me.

I believe the last idea you have in your post is a joke, and I won't bother addressing it. I hope you don't mind

EDIT: I think search by guests is disabled by default to prevent DOS attacks. Search is also rate-limited for users.

I insist that user should be able to lurk without having to ping back every second week. Some people might want to be able to send and receive private messages (if I want to contact someone I found on a server, the first place I would look to contact them would be the forums). Requiring one year of inactivity before blocking then removing the account seems reasonable to me.

I'm not suggesting that any 2 weeks of inactivity equals account closure. What I'm suggesting is that if a user makes a new registration and within 2 weeks does not post, they should be emailed reminding them that they need to post or else their account is closed. Once they have a post, this check never applies again.

This works fine for the non-active player pickup suggestion posted as it is entirely constructive for such players to post in the Oh Hi section introducing themselves, what games types they play and on what servers and hence their accounts would be active and they would be outside of the probationary period.

Sleeper spambot accounts would be picked up unless they post in which case they would be more obvious to deal with.

Hm. Good point. Do you think the mandatory post should be a thread in the Oh Hi section?

Hm. Good point. Do you think the mandatory post should be a thread in the Oh Hi section?

I wouldn't make it specific to Oh Hi. The other major reason for people to join the forum is that they have some problem and you don't want people who can't get the game working to first have to introduce themselves. The Oh Hi forum would get filled with attached technical queries and assorted whinges and moans. Hence it should be a general requirement of a single post in any forum that they want.

If they just want to say Hi, that's fine, if they want to ask a technical question, that's fine or go straight to the special corner that is WW, that's also fine, they just need to do something that shows they're out there and real.

Good points again. Thanks!

Mr. Bougo, i could help with manyal huntt on spambots.

By the way, I think that removing spammers' acounts entirly would be a better idea.

I don't purge the spambot group because it contains data. As I said above we can use that to find good strategies against them.

Two suggestions to handle this problem:

1. Make ALL boards invisible for non-registered people
2. Make ALL profiles invisible for non-registered people

You have the downside of search engines not being able to index the boards anymore ... but in the end this is just what you want, because then spammers would be only wasting their time on this boards. No seach engine indexing, no profit, no spam.

I know this is a kind of "radical" solution, but IMHO its the only solution that will really work out.

You can btw kind of overcome this problem of not being indexed anymore using the "archive mode". Just make sure to remove the links to profiles in those templates and remove the displaying of signatures. If you then implement an URL-filter you are all set up as no spambots would be able to profit from the boards anymore and you still dont have the downside of not being indexed. Furthermore you could add some little Javascript code to redirect users from the archive mode to the full mode, when they enter the boards from a search engine in archive mode.

Im pretty sure now that some of you may dislike my idea ... but let me tell you this: Radical problems need radical solutions

Hiding profiles seems like a fine idea. Hiding the boards will never happen, though.

1. Make ALL boards invisible for non-registered people

In that case you will have people having to register just to read the forums which then means more inactive accounts. People will forget their passwords mroe if they are only setting up an account to read the forums rather than post and so there will be far more forum registrations with 0 posts. That then makes it harder to tell the difference between spammers and users.

Blocking viewing profiles for non-registered users is good though.