Jul
8

Brainbench offering FREE tests!

Posted by Aaron West at 3:30 PM in General

I heard about Brainbench offering free tests in celebration of their 5,000,000th member. I thought it was a hoax until I received an invitation from Brainbench (as a current member) to take part in the free tests from July 1 - July 14th. I cruised over to Brainbench today during lunch and took two tests: ColdFusionMX and DreamweaverMX. I had been wanting to take the CFMX test for obvious reasons but being the cheapstake I am I didn't want to pay for it. It wasn't too bad and I passed scoring better than 91% of previous test takers. I had some more time so I decided to attempt the DWMX test. I don't really use DWMX much (I'm a HomeSite+ guy!) so I didn't expect to do well. In fact, I guessed on most of the answers through my own process of elimination. Suprisingly, I passed scoring better than 93% of previous test takers. 93%???? How is it possible for me to score better than 93% when I know so little about DWMX. Either the previous test takers need to seriously bone up or I'm one incredible test taker.

Either way, I have two more "certifications" banners on my Contact Info page.

Aaron West's Gravatar
About this post:

This entry was posted by Aaron West on July 8, 2004 at 3:30 PM. It was filed in the following categories: General. It has been viewed 6884 times and has 2 comments.

2 Responses to Brainbench offering FREE tests!

  1. Ralph

    On November 15th of 2006, I took the Brainbench Java 2 test.

    When I went into the test, I was expecting something that would test my general knowledge of the Java language
    and object oriented programming. The test that I took did not do that.

    There were a large number of questions on special purpose API's that I have never used. There were some
    questions on development tools that I have never used. And there were a large number of "brain teaser"
    questions on code snippets which I could have answered, if I would have had more time.

    As a result, my test score was very low (2.60).

    That score shows that I'm not good at guessing at API's and tools that I've never used. And it shows that I'm not
    good at brain teasers with a three-minute time limit. However, it shows ABSOLUTELY NOTHING about my Java
    and object oriented programming skills.

    In 2002, I studied a book on Java, and then I took a college course on it. I got an A in the course. According to
    my Brainbench score, I should not have been an "A" student.

    For over a year, I was the sole designer and developer of web software that my employer believes is marketable.
    That software involves thousands of lines of Java code, a large number of API's, and a large number of advanced
    object oriented constructs. According to my Brainbench score, I should not be able to accomplish what I have just
    accomplished.

    When I was in college, I took secondary education courses. Later, while I was working for a former employer, I
    designed two computer courses which I taught for several years. As a trained and experienced trainer and tester,
    it is my opinion that the Brainbench Java 2 test has a number of very serious flaws.

    It appears that there have been no sound scientific studies regarding the Brainbench claim that their tests predict
    employee success. And it appears that there are a large number of companies that are blindly accepting these
    unsubstantiated claims.

    It seems to me that a sound scientific study for the Java 2 test would include the following elements: have
    thousands of working and successful Java programmers take various Java 2 tests; have thousands of
    inexperienced people with Java knowledge take the same tests; for individual test takers, have tests with a large
    number of questions on API's and tools that they have never used; for individual test takers, have tests with a
    large number of questions on API's and tools that they have used; for the latter, follow their careers as Java
    programmers for at least five years.

    It appears to me that the creators of the Brainbench Java 2 test do not know what a typical Java programmer
    does, and they have no understanding of the art of testing.

    I wonder how many careers have been derailed as a result of flawed Brainbench tests. I would like to see a
    scientific survey on that.

  2. I'm on the fence when it comes to the effectiveness of certification testing. As I mentioned in my post (wow, it's been over two years since posting it) I found the CFMX and DWMX tests to be quite easy. You found the Java 2 test to be difficult. At least, difficult in terms of getting a passing score. Does that mean either test is ineffective? Maybe. Does it matter? To some yes, and to others, no. It may sound like I'm non-committal but there are good tests and there are bad tests. There are also good test takers and poor test takers. The takeaway is this: judging someones technical ability solely on how they perform on a technical test means you discount all other factors that make up the landscape of someones skill-set and experience therein. When hiring developers for my team I do administer programming and logic tests but these are not the only inputs in my decision making process. I conduct phone screens, in-person interviews, reference checks and have members of my team interview candidates as well.

    All this to say, I wouldn't worry about it. If you miss out on a job because the hiring manager sees your Java 2 score, you probably don't want to work for that company anyhow. Furthermore, I personally question the accuracy of some of Brainbench's tests in regards to really really testing technical knowledge. I've taken several of Brainbench's tests and several Adobe / Macromedia certification tests. The latter of which do a vastly better job and thoroughly testing candidates. I give very little weight to BrainBench certifications and a good bit of weight to Adobe certifications. Of course, knowing candidates even attempted to get certified tells me they more than likely take their career seriously. One last thought regarding your comments on Java API's... it is very, very difficult to test the full spectrum of a senior level developers abilities. Especially in a language like Java that, as you indicated, has dozens if not hundreds of API's. It would be difficult for you as a developer to thoroughly understand and have experience in them all. And it would be equally difficult for test writers to test you on them all. Focusing on the most common language functions, fundamentals, API's, and programming concepts is, in my opinion, the recipe for creating a valuable test.