Spam and forum explosion

[Eric]

To start off, I'd like to thank you guys for your patience over the past couple days.  The extended downtime was 100% my fault. I always create a database backup for the whole site before performing an upgrade.  I always do a manual software upgrade too, and this time I figured I'd give the automatic one-click upgrade feature a try.  
 
You know that feeling you get just as you realize that something bad has happened and it's about to cost you a lot of time, and it all could have easily been avoided?  Yeah, about 1/3 the way through the software upgrade, some nasty database errors show up, and after pounding my fists on my desk like a child, I got to work.
 
In case you missed the updates while the site was down, here they are:

Website Maintenance
 
It looks like the spammers finally found the answer to our security question during registration (four digit number associated with an income tax return), allowing them to create a bunch of accounts and start posting spam like crazy.

 

I'm in the process of cleaning up the spam accounts and removing the spam posts right now, but it will take some time. I'll add some new security questions to the registration form as well.
 
Update: Dec 14 - 1:00PM (eastern time)
In other news, a forum upgrade hosed the forum database. I was in a hurry and didn't perform a backup before the installation, so it's going to take some time to fix. I have a support ticket in with the software vendor, which has been escallated to Tier 2 support. That's what I get for being in a hurry an idiot.

Update: Dec 14 - 3:00PM
From what I can tell, none of the important data (member info, posts, etc) have been affected. Tier 2 support probably isn't available on the weekend. I'll post updates here as new information becomes available.

Update: Dec 15 - 10:30AM
Not much of an update. Still waiting to be contacted by Tier 2 Support from the vendor. Just wanted to let you know that I hadn't forgotten about you

Update: Dec 15 - 1:15PM
The Database issues have been resolved, and now I can resume working on the spam issues. Shouldn't be too much longer. Thank you all for your patience.

 

 

And now here we are.

 

So welcome back everyone, I have learned my lesson about skipping database backups.  

 

PS - To anyone who registered over the past couple days, I apologize, your account has been deleted along with the 80+ spam accounts that were registered during the same time frame.  We don't normally get more than 1 or 2 daily registrations, so rather than inspecting every single new account's IP address, I just deleted them all.  You'll need to register again.

[Eric]

Also, if there are any suggestions for new tax related security questions that are asked during registration, send them to me in a private message.

 

A good security question is:

• One that a tax preparer would find obvious
• Preferably something that is specific to tax preparation, and not general knowledge in Russia which is where a lot of these spam accounts come from.
• One that can only be answered only one way, or only a few ways.  It would be obnoxious to be asked a question, only to be told that your answer is wrong because you worded it slightly differently.  Answers given must match my pre-defined answers EXACTLY for them to work.  Answers that are common abbreviations, or numbers work well

[Richcpaman]

Which IRS form Number does the client sign for you to file them Electronically?

Edited December 15, 2014 by Eric
Edited out the answer

[NECPA in NEBRASKA]

How about asking what form does the taxpayer have to sign to efile a 1040?

Edited December 15, 2014 by Eric
Edited out the answer

[NECPA in NEBRASKA]

Apparently, great minds think alike. We posted that at the same time.

[Eric]

I'm going to hide any responses given here, so answers aren't easily noted by spammers.

[mcb39]

Kudos to you Eric.  That was really a mess on Sunday morning.    I could hardly believe my eyes and tried to PM you as well as the moderators.  You probably already knew what had happened.  I am sure that nobody on this board is going to hold you to blame in any way.  Thanks for the quick action.  You can beat up on yourself if you want to, but we are so glad and lucky to have you.

[jklcpa]

Eric, thanks for all of your hard work and your patience as well.  I sent you a PM with some suggestions for security questions and answers too.

[Lion EA]

Just glad to have you on our side.  Thanks!

[DevM]

Great Job!!! Thank you very much.

 

I just would like to recommend a little donation for Eric for his great effort.

[JohnH]

s

^{Great Job!!! Thank you very much.}

 

^{I just would like to recommend a little donation for Eric for his great effort.}

 

^{I agree 100%.}

^{Just gladly & gratefully sent mine - highly recommend that others do the same.}

[jklcpa]

After disabling about a dozen last night before signing off, and then seeing Eric's message this morning about more than 80 (!) new users and about all of the spam, it was wonderful to see those IP addresses trying and not being able to log on when Eric opened the site back up to us.

 

Well done, Eric!

[Eric]

Got the site all connected up to an anti-spam service provided by the forum software vendor, and it seems to be working.  Three attempted registrations auto-banned from the site in the first few hours after the site came back online.

 

NO SPAM FOR YOU. ONE YEAR.

[Jack from Ohio]

Got the site all connected up to an anti-spam service provided by the forum software vendor, and it seems to be working.  Three attempted registrations auto-banned from the site in the first few hours after the site came back online.

 

NO SPAM FOR YOU. ONE YEAR.

How much additional costs did you incur for this service?

[Eric]

How much additional costs did you incur for this service?

 

The service itself doesn't cost anything, as long as I keep the software subscription up to date.  I tend to let mine lapse until there's an available software update, but now I'll just keep it current.  I guess I can kind of compare it to Maine Vehicle Inspection.  You're supposed to do it once a year, but if you do it a couple months late every year, at the end of 6 years it's like a free inspection!  (which I do because I'm good at procrastination, not because I'm trying to save $14 every 6 years)

 

In other words, it's not much of a financial impact.

[JohnH]

But what about the financial impact on your time?

After all, you continually spend hours working on this forum which could be spent with your family, reading, hiking, skiing, taking a nap, etc.

[Eric]

In this specific case, had I been doing work for any one of my clients, there is no way I'd bill them for the time I spent fixing my mistake.  In fact, I would have refunded a month's hosting and maintenance as a result of the downtime.

 

I do appreciate the donations, you guys are a very generous bunch.  Thank you.

[Pacun]

I have a question, when I report someone, who gets the message?

 

I sent like three reports as soon as I saw the spam and I wonder who got it and if there is an easy way of doing it.

[Eric]

Mods and Admins get the reports, but everyone has their own notification settings.  Personally, I have mine set to send me an email notification when someone reports content.

[jklcpa]

Pacun, I saw the start of all the new users being set up between around midnight and 3 am and disabled the posting functions of those that I'd found, and I could tell that they were foreign-based users and IP addresses. At that point, there was only one real spam posting that wasn't offensive in nature but had a ton of links in it.  I left Eric notes in the moderator section about what I'd done and why, and after I'd signed off in those next few hours is when it really exploded with all the additional users, and by morning Eric saw members making reports and shut the forum down completely to work on the problem.

 

KC and I do see those reports also. Previous to this, I had my report setting to not send to my general email box because I leave a tab open and check in frequently during the day, as frequently as I check my emails for sure. Some of the functions available to KC and I as moderators are the ability to modify or delete posts, and to manage members in certain ways including disabling posting and eliminating spammers one user at a time, but this issue was well beyond what either of us would have been able to fix. 

[Pacun]

Thank you for the explanation. I am glad it is over.

[kcjenkins]

It's not the first time, and likely not the last, but we will always take action when it happens.  

[ILLMAS]

Would making the site only available to sign-in members prevent from other people taking a look at the discussions and deter them from spamming or creating an account?

[jklcpa]

Would making the site only available to sign-in members prevent from other people taking a look at the discussions and deter them from spamming or creating an account?

 

The spammers don't care what the topics are or sites' main interests. When I started looking into where these were generated and searched for some of them with whois, the IPs came up as spammers that were signed on to a wide variety of sites for the sole purpose of spamming, and some were reported to be associated with blacklisted sites. This forum's internal IP tools revealed that some of the members had multiple members all signing in from the same static IP address, and those might have been automated spam bots being set up once a human had solved the old security question.

 

The new sign up features that Eric put in place on Monday assure that new members are real people and whose IP addresses are not associated with high levels of spam.

[Randall]

The Form number suggested for a security question seems like it might be easily determined by the spammers.