Awhile back I came upon term selenium and even took some time to check it out and make note to use it later. For those of you who dont know; Selenium is a tool for automating website and application testing. This morning I got my mac setup with homebrew, maven, eclipse (already setup), java (already setup), selenium, and safariDriver. I was able to get the dev enviornment and a test app working in under an hour, with most of the time watching a horribly slow “uhm uh” youtube video with minutes of over explanation. From the example I now have a java app that will open my website, type a term into the search box, and push the search button.
Why do we need to automate website testing? I am getting ready to start testing a new product page on my shopping cart. The new functionality is mostly basic for some large percentage of the inventory (3,000-5,000 products – 30-50,000 BarCodes). However, some small percentage, has data situations that are not so predictable, or even known. These may even be product with incomplete or missing data that will be required to add to cart with the new product page. Basically the old needle in a haystack.
We had discussed tasking employees with the job of making sure all of the product they are responsible for is working. However, we all know in the real world the real testers will be the users and customers, and if they cannot add to cart, they will not send a report. So the goal of this blog post will be to chronicle the building of an application capable of visiting every product page, checking out all available options per product, and providing data to find product not able to be added.
 Later in the afternoon after starting this blog post I quickly got the mysql java connector working in my application. I was very easily able to get it to query the first 50 product, visit each page, and add the item to the shopping cart. At the end of the test my cart has over 50 items in it. I did have an issue with other user interaction popups (free gift) stopping automation flow but these things are to be expected. My automation is not smart enough yet but it will be soon.
Now that I can add to cart, the next best thing is obviously to check out a 50 item order. This will take some deeper actions, but should be very easy to create the code. To do this I need to load the cart, click checkout, enter payment information, click continue to order view, then finally submit the order. Since I am using my native dev browser I am already logged in with a cookie. Later I will need to add login/register type tests as well.
This was so easy pretty soon I am doing 50 item orders very easy with just 65 lines of code. Just for fun I am now testing 500 item order. Start Time : 3:32. In order to get the test working, I had to have 1 item in cart with free gift already added, then it would add 500, so total cart items should be 503. Finish Time: 4:04 with 501 total items in the cart.
My next focus needs to be making my automation smart enough to be more useful and better handle exceptions, errors, and timeouts.
 For my next Selenium sitting, I focused on handling the Free Gift. This took a bit to make sure that the modal was actually open and visible. The modal is always in the page so for this purpose we only add the free gift if the modal is visible (cart over $99.99). Once I had the free gift working I started comparing the total products added versus total products in the cart. With free gift working when I run the automation test for 5 products I should have 6 in my cart. When I run the automation test for 10 products I should have 11 in my cart. With this basic logic working I then added a conditional if the total is correct, the order is then checked out. I checked out 10, 10, and 50 product orders without any issues. I then turned it on for 500 and I will check back in about 30 minutes. 503 Items again and dev store is loading so slow it did not checkout. Where are these 2 extra items coming from??
My next focus needs to be how to handle timeouts and total item exceptions.
 This morning I decided to address issues with loading such large shopping cart pages and less on total # comparisons. The dev site was pretty slow yesterday and loading a cart with 500+ line items was taking too long. I also had a hard time seeing where the extra items were coming from. While inspecting the cart everything looked to be in order. I did not find any free cart items, coupon items, etc. It is possible one of the items is a package or something where adding 1 product puts 2 products in the cart. To solve both of these things I am going to adjust my automation test to checkout when the products hit 50. So for a 500 test i should expect 10 orders of 51 items. Each 50 items + a free gift. This will reduce the page loading times during checkout as well as stop in a smaller data set when the quantity doesnt match.
Turns out the site being slow was an issue with digital ocean NY2 Node (web) communicating with NY3 Node (db). Nice. For now I have set the test site to run on localhost and the site loads instantly again for my needs. After some time working with localhost, I went back to work on correct db. I made some adjustments to the automation test but still was struggling with the network latency issues where an expected page element does not exist.
I also took some time to add in a comparison with the total carts added and the total carts in the database ready to order. If these match the order is completed. When the item exists in the cart, I flag a field (automated) in the database. I then adjusted the main query for items so that it only grabs the unflagged products. After running this automation test for 500 total products 5 or 6 times I should have a majority of the inventory in the cart as ordered.
My next focus will then be finishing the execution of all the basic products and preselected optionable skus.
 This morning I increased the staged sleeps within the automated steps a great deal and was able to get pretty good success. Slow and steady wins the race. My goal is to get through the entire database of main products(~3,500 products) with as many of them flagged automated as possible. I will then be able to focus on the product data exceptions and issues.
Even with timeouts from 2000 to 6000 this is not an actual guarantee that an element in the page has loaded or visible. A better usage would be to wait in a loop until element exists. I tried a few of the known timeout/wait methods and nothing really seemed rock solid. Even after making a driver timeout load up to 10 seconds, sometimes the freegift modal element still was not ready. I had to add another 6 second wait to check for free gift until it is added. With larger tests this should be less of an issue as free gift is added very fast. After some adjustments I was able to get 25 products through at 5 products per order.
This point of the program is saved off in favor of moving faster.
For larger testing I created a separate scripts for creating the cart and checking out the cart. Once the checkout was marking the items properly flagged as automated I am able to do larger carts, and eliminate testing so many orders or free gift loading and not loading. I can stop this script at any time and then run the other one to checkout. Here I am really interested in getting all the basic products, or basic skus added to the cart, through to an order, and flagged as automated. The more items I run through automation, the lower the results are that are NOT added to cart and that is the goal.
My next focus will need to be working with products that have SKUS that are not ordered in the cart and any changes I need to make in the automation to select those skus. Once this is ready I should be able to eliminate another major portion of the un-automated data.
 By end of the day yesterday i had the automation test going for 250 products and an order quite a few times (5+ times). I did another run this morning and then decided to do pickcolor, picksize, and pickoption. This was very easy, too easy in fact. First I did necessary code to click all the colors and add then click add to cart. This was very easily changed for size and option to get them all working. I also has to make it select all the sizes after selecting a colors as it is possible to have a color with multiple sizes.
The only weird issue I had for this sitting was color picks that were hidden by owl-carousel causing the webDriver to crash. There was an internal topic about how this scroller can hide some colors. If a user did not know to scroll to the right they might not ever see them. This issue was a strike 2 & 3 for me. I removed the carousel from the color pick area and set some static CSS to make the thumbnails smaller and remain inline. With this sorted I am able to run the automation test and add all of a products skus before going to the next product page.
My next focus will need to be deeper testing (shoes & advanced attribute option picks) and preparing to flush all the automated tags, and run again from start.
[6 & 7 ] For the last 2 days I have been running the automation and making very minor adjustments based on any stop points or any trouble within the product page itself. After these adjustments the automated test is working really good. A large selection of shoe product pages automated without any issues . I only had to make some changes to the product page to adjust how shoe sizes were handled. The automation will now select each shoe color, each shoe size and then add them to the cart. If some shoe size is not in stock for a particular color, it is disabled, not operational, and skipped for add to cart.
I am currently showing about 9,000 total products completed and automated. There are less than 600 total master products that have not been through the automation yet. I expect to be done with those today and be able to see what is left to focus on those.