DbTradeAlert for Android: Automated UI Tests – Part 2: Testing with Espresso

First post in this series: Introduction to DbTradeAlert

Previous post: Automated UI Tests – Part 1: Thoughts about Testing

This post has two additional requirements in regards to the things set-up so far:

  • Android Studio 2.2 or later is needed for the Espresso test recorder
  • Firebase Blaze plan is needed for access to Firebase Test Lab for Android – the post describes the upgrade process

Also note that this Espresso test will be somewhat special as it requires a human to decide whether the scrolling was smooth. Espresso is capable of recording a fully automated UI test by just clicking on a screen element to add an assert about its contents.


5. Automated Scrolling Experience Test

The scrolling experience with larger watchlists on low end devices was identified to be a concern due to the lack of physical devices to test on. Firebase Test Lab for Android provides those to run for example an Espresso test.

5.1 Record an Espresso Test

Recording an Espresso test is not as straight forward as one would think because the test recorder is still in beta. I choose to list every error message thrown at me in the process because those problems will be fairly common.

Let’s record a swipe test first:

  1. In Android Studio select Run | Record Espresso Test
  2. In the Select Deploymet Target window select your device or emulator and click OK
  3. Wait until the Record Your Test window shows up and the app is ready
  4. Swipe up – see comments if that doesn’t get recorded
  5. In the Record Your Test window click Complete Recording
  6. In the following dialog enter a class name for the test – “ScrollingExperienceTest” in my case – and click Save
  7. Accept the offer to add missing dependencies

That should leave you with an automated UI test ready to replay. But as Espresso test recorder is still beta it leaves some manual work for you.

First, make the ScrollingExperienceTest class compile:

  • Fix the R class access – replace “de.dbremes.dbtradealert.withAds.R.” with “R.” in my case
  • Let Andoid Studio add a static import for matchers like withId()

More importantly the swipe wasn’t recorded because while Espresso supports swipes its test recorder doesn’t as of version 2.2.2. To have it at least build the infrastructure I long-tapped a report which got recorded properly. Then I just coded the swipe manually and added the necessary static imports:

package de.dbremes.dbtradealert;

import android.support.test.espresso.ViewInteraction;
import android.support.test.rule.ActivityTestRule;
import android.support.test.runner.AndroidJUnit4;
import android.test.suitebuilder.annotation.LargeTest;

import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;

import static android.support.test.espresso.Espresso.onView;
import static android.support.test.espresso.action.ViewActions.click;
import static android.support.test.espresso.action.ViewActions.swipeLeft;
import static android.support.test.espresso.action.ViewActions.swipeUp;
import static android.support.test.espresso.contrib.RecyclerViewActions.actionOnItemAtPosition;
import static android.support.test.espresso.matcher.ViewMatchers.isDisplayed;
import static android.support.test.espresso.matcher.ViewMatchers.withId;
import static android.support.test.espresso.matcher.ViewMatchers.withParent;
import static android.support.test.espresso.matcher.ViewMatchers.withText;
import static org.hamcrest.Matchers.allOf;

@LargeTest
@RunWith(AndroidJUnit4.class)
public class ScrollingExperienceTest {

    @Rule
    public ActivityTestRule<WatchlistListActivity> mActivityTestRule = new ActivityTestRule<>(WatchlistListActivity.class);

    @Test
    public void scrollingExperienceTest() {
        ViewInteraction recyclerView = onView(
                allOf(withId(R.id.list),
                        withParent(allOf(withId(R.id.container),
                                withParent(withId(R.id.main_content)))),
                        isDisplayed()));

        recyclerView.perform(actionOnItemAtPosition(0, swipeUp()));

        // Freezing the app to see the swipe
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

To run the test:

  1. In the test method’s context menu select Run scrollingExperienceTest
  2. In the Select Deploymet Target window select your device or emulator and click OK

The test will probably fail with this error message:
PerformException: Error performing 'fast swipe' on view. Animations or transitions are enabled on the target device.

That’s because Espresso can only detect whether the app itself is busy but not when animations or transitions are running. That may lead to tests failing for the wrong reason and Espresso prevents that. Disable animations on the device for Espresso to work:

  1. Go to Settings | Developer options
  2. Disable 3 settings:
    1. Window animation scale
    2. Transition animation scale
    3. Animator duration scale

Running the test again produced this error on the withAds flavor:
java.lang.RuntimeException: Action will not be performed because the target view does not match one or more of the following constraints:
at least 90 percent of the view's area is displayed to the user.
Target view: "RecyclerView ...

That’s because the advert covers part of the RecyclerView. So let’s ty the playStore flavor.

Running the test on the playStore flavor produced this error – buried deep in the stack trace:
android.support.test.espresso.InjectEventSecurityException: java.lang.SecurityException: Injecting to another application requires INJECT_EVENTS permission

Hmm, head-scratching after this one. But a simple solution after finally checking the originally recorded long-tap – perform the action on an item and not on the RecyclerView itself. Just replace
recyclerView.perform(swipeUp()) with
recyclerView.perform(actionOnItemAtPosition(0, swipeUp()))

Adding a Thread.sleep(1000) permits to see if the swipeUp() actually moves the watchlist.

5.2 Generate Test Data

The app starts with two watchlists each containing two securities. As the idea is to force the creation of new reports while scrolling the app needs more securities. Logging will be added to make shure the reports weren’t created in advance.

To add the required securities to the watchlist the test will read them from a .csv file. The DbHelper class has already code to parse .csv data and the infrastructure will come in handy later when using WireMock to provide canned quotes.

5.2.1 Add a CSV File to the Test

The .csv file is only used for testing and therefore shouldn’t be included in the app itself. Instrumented tests like Espresso tests actually get compiled into a second apk that runs in the same process as the app’s APK. You’ll see that in Andoid Studio’s test window:
$ adb shell pm install -r "/data/local/tmp/de.dbremes.dbtradealert.playStore"
...
$ adb shell pm install -r "/data/local/tmp/de.dbremes.dbtradealert.playStore.test"

The first step is to create an assets folder under “<ProjectPath>\app\src\androidTest” with Explorer as Android Studio doesn’t support that.

After that use Android Studio to create the .csv file:

  1. In the “app” context menu select New | File
  2. In the Choose Destination Directory window select “…\app\src\androidTest\assets” and click OK
  3. In the New File window enter “ch_securities.csv” and click OK
  4. In the Register New File Type window accept the default of opening Text files in Android Studio and click OK

The contents can be added in any text editor:

"ABBN.VX"
"CFR.VX"
"ROG.VX"
"SYNN.VX"
"UBSG.VX"
"ZURN.VX"

5.2.2 Import CSV Data

Importing CSV data with DbHelper is straight forward:

// ...
public class DbHelper extends SQLiteOpenHelper {
    // ...

    private String[][] convertCsvToStringArrays(String csvData) {
        String[][] result = null;
        String[] csvRows = csvData.split("\r?\n|\r");
        int rowCount = csvRows.length;
        int fieldCount = csvRows[0].split(",").length;
        result = new String[rowCount][fieldCount];
        String csvRow = null;
        for (int rowIndex = 0; rowIndex < csvRows.length; rowIndex++) {
            csvRow = csvRows[rowIndex];
            result[rowIndex] = csvRow.split(",");
            // Delete any surrounding quotes
            for (int fieldIndex = 0; fieldIndex < result[rowIndex].length; fieldIndex++) {
                result[rowIndex][fieldIndex] = result[rowIndex][fieldIndex].replace("\"", "");
            }
        }
        return result;
    } // convertCsvToStringArrays()

    // ...

    public void importTestSecurities(String csvData, long watchlistId) {
        final String METHOD_NAME = "importTestSecurities";
        final String addSecurityToWatchlistSql = "INSERT INTO " + SecuritiesInWatchlists.TABLE
                + "(" + SecuritiesInWatchlists.SECURITY_ID
                + "," + SecuritiesInWatchlists.WATCHLIST_ID + ") VALUES(?,?)";
        final String insertSecuritySql = "INSERT INTO " + Security.TABLE
                + "(" + Security.SYMBOL + ") VALUES(?)";
        String[][] csvArrays = convertCsvToStringArrays(csvData);
        SQLiteDatabase db = this.getWritableDatabase();
        try {
            db.beginTransaction();
            SQLiteStatement addSecurityToWatchlistStatement
                    = db.compileStatement(addSecurityToWatchlistSql);
            SQLiteStatement insertSecurityStatement = db.compileStatement(insertSecuritySql);
            for (int rowIndex = 0; rowIndex < csvArrays.length; rowIndex++) {
                // 1-based index for bindString()!
                insertSecurityStatement.bindString(1, csvArrays[rowIndex][0]);
                long securityId = insertSecurityStatement.executeInsert();
                addSecurityToWatchlistStatement.bindLong(1, securityId);
                addSecurityToWatchlistStatement.bindLong(2, watchlistId);
                addSecurityToWatchlistStatement.executeInsert();
            }
            db.setTransactionSuccessful();
            Log.d(CLASS_NAME, METHOD_NAME + "(): success!");
        } finally {
            db.endTransaction();
        }
    } // importTestSecurities()

    // ...
}

The new importTestSecurities() method has two duties: parse the csv data and insert the resulting values into the securites table which includes adding them to a watchlist.

Parsing the csv format was already implemented in updateOrCreateQuotes() and got factored out into convertCsvToStringArrays() which returns a two dimensional string array.

Inserting the values happens a bit more performance-conscious here than in updateOrCreateQuotes(). As before the main booster is to wrap the inserts in a transaction – that cuts 99% off non-transactional execution time easily because it saves SQLite from needing to wrap each in its own transaction.

What’s new is the call to SQLiteDatabase.compileStatement(). This way preparation steps like analyzing the SQL are only executed once which can cut the remaining execution time in half. An SQLiteStatement absorbs parameter values with type specific methods like SQLiteStatement.bindString() – note the 1-based index and of course you have to clear a parameter value if you don’t provide a new one. The execution method to call – SQLiteStatement.executeInsert() in this case – is specific to the type of SQL statement. Unlike other databases SQLite provides no way to unprepare a statement to free its resources.

To be honest I wouldn’t use prepared statements for the few records this test inserts but just wanted to try them with SQLite. If on the other hand you have to insert tens of thousands of records it may be worth doing that in chunks to prevent memory issues.

SQlite actually supports bulk imports – loading data directly from a file – with “.import <FILE> <TABLE>” but only at the command line interface. And finally starting with version 3.7.11 it provides this insert syntax:
INSERT INTO 'tablename' ('column1', 'column2') VALUES
('data1', 'data2'),
('data3', 'data4'),
('data5', 'data6'),
('data7', 'data8');

Note that this is only syntactical sugar without performance benefits and certainly non-standard SQL.

5.2.3 Start Data Import Before the Test

To start the data import ScrollingExperienceTest gets a new method:

// ...
@LargeTest
@RunWith(AndroidJUnit4.class)
public class ScrollingExperienceTest {
    // ...

    @Before
    public void createTestData() {
        String testDataString = "";
        Context testContext = InstrumentationRegistry.getContext();
        try {
            InputStream testDataStream
                    = testContext.getResources().getAssets().open("ch_securities.csv");
            byte[] testData = new byte[testDataStream.available()];
            testDataStream.read(testData);
            testDataString = new String(testData);
        } catch (IOException e) {
            PlayStoreHelper.logError(e);
        }
        Context targetContext = InstrumentationRegistry.getTargetContext();
        DbHelper dbHelper = new DbHelper(targetContext);
        // SQLite Ids start with 1
        long watchlistId = 1;
        Log.d(CLASS_NAME, "createTestData(): calling importTestSecurities()");
        dbHelper.importTestSecurities(testDataString, watchlistId);
    } // createTestData()

    // ...
}

The @Before annotation tells JUnit to execute createTestData() before running any test. The method reads the .csv file from the test package’s resources – large files should be handled in chunks. Also note that createTestData() uses two different contexts:

  • InstrumentationRegistry.getContext() provides the test app context to access the file in the respective package
  • InstrumentationRegistry.getTargetContext() provides the app context to create a DbHelper instance

Another thing to note is that SQLite’s automatic identifiers are 1-based.

5.3 Delete Test Data After the Test

Every test should leave the app in the same state it was before. That’s the only way to ensure a defined starting state when running multiple tests.

The easiest way is to wrap the whole test beginning with test data creation in a transaction and roll it back after test completion. Of course this can lock surprisingly large parts of the database and should be used with caution.

SQLite doesn’t support nested transactions out of the box but uses savepoint and release commands. The SQLite wrapper for Android maps that to nested transactions though.

But that still doesn’t work for DbTradeAlert’s stateless DbHelper class. The problem is that nested transactions in SQLite have to use the same SQLiteDatabase instance. As DbHelper’s methods create a new SQLiteDatabase instance whenever they need one tests cannot use a transaction without changing DbHelper’s API. Trying it with SQLiteDatabase.beginTransaction() will result in a SQLiteDatabaseLockedException and if you use SQLiteDatabase.beginTransactionNonExclusive() you’ll get a IllegalStateException later – “no transaction pending”.

So DbHelper gets a new method instead:

// ...
public class DbHelper extends SQLiteOpenHelper {
    // ...

    /**
     * Caution: deleteTestSecurities() deletes any securities with Id > 4!
     * Logic as deleteSecurity()
     */
    public void deleteTestSecurities() {
        final String methodName = "deleteTestSecurities";
        Log.v(CLASS_NAME, String.format("%s(): securityId > 4", methodName));
        String[] whereArgs = new String[]{String.valueOf(4)};
        int deleteResult = 0;
        SQLiteDatabase db = getWritableDatabase();
        try {
            db.beginTransaction();
            // Delete security's quotes
            deleteResult = db.delete(Quote.TABLE, Quote.SECURITY_ID + " > ?", whereArgs);
            Log.v(CLASS_NAME, String.format(DELETE_RESULT_FORMAT, methodName,
                    Quote.TABLE, deleteResult));
            // Delete security's existing connections to watchlists
            deleteResult = db.delete(SecuritiesInWatchlists.TABLE,
                    SecuritiesInWatchlists.SECURITY_ID + " > ?", whereArgs);
            Log.v(CLASS_NAME, String.format(DELETE_RESULT_FORMAT, methodName,
                    SecuritiesInWatchlists.TABLE, deleteResult));
            // Delete security
            deleteResult = db.delete(Security.TABLE, Security.ID + " > ?", whereArgs);
            Log.v(CLASS_NAME, String.format(DELETE_RESULT_FORMAT, methodName,
                    Security.TABLE, deleteResult));
            db.setTransactionSuccessful();
            Log.d(CLASS_NAME, methodName + "(): success!");
        } finally {
            db.endTransaction();
        }
    } // deleteTestSecurities()
 
   // ...
}

This new method called deleteTestSecurities() implements the same logic as deleteSecurity() but does so for all securities with Id > 4. This way you can repeat the test without causing a UNIQUE constraint violation when adding the same symbols again. As the tables were created with INTEGER PRIMARY KEY columns and not as AUTOINCREMENT the Ids get recycled.

This method is called from ScrollingExperienceTest:

// ...
@LargeTest
@RunWith(AndroidJUnit4.class)
public class ScrollingExperienceTest {
    // ...

     @After
    public void deleteTestData() {
        Context targetContext = InstrumentationRegistry.getTargetContext();
        DbHelper dbHelper = new DbHelper(targetContext);
        dbHelper.deleteTestSecurities();
    } // deleteTestData()

    // ...
}

As the method is annotated with @After it gets called automatically after the tests have completed.

5.4 Run the Test

Actual test methods like scrollingExperienceTest() are marked with the @Test annotation. In this case the generated method needs an overhaul before it can do anything useful:

// ...
@LargeTest
@RunWith(AndroidJUnit4.class)
public class ScrollingExperienceTest {
    // ...

    @Test
    public void scrollingExperienceTest() {
        Log.d(CLASS_NAME, "scrollingExperienceTest(): start");
        // Make added securities show up in watchlist:
        Log.d(CLASS_NAME, "scrollingExperienceTest(): tapping Refresh");
        ViewInteraction actionMenuItemView = onView(
                allOf(withId(R.id.action_refresh), withContentDescription("Refresh"), isDisplayed()));
        actionMenuItemView.perform(click());

        try {
            Thread.sleep(3000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        ViewInteraction recyclerView = onView(
                allOf(withId(R.id.list),
                        withParent(allOf(withId(R.id.container),
                                withParent(withId(R.id.main_content)))),
                        isDisplayed()));
        for (int i = 0; i < 8; i++) {
            Log.d(CLASS_NAME, "scrollingExperienceTest(): swipeUp" + i);
            recyclerView.perform(actionOnItemAtPosition(i, swipeUp()));
        }
        Log.d(CLASS_NAME, "scrollingExperienceTest(): the End");
    } // scrollingExperienceTest()

    // ...
}

The first necessary change is to tap the Refresh button because the new securities will show up only after they got their quotes. Currently the quotes are actually loaded from the internet and a Thread.sleep(3000) makes shure they arrived before continuing the test – this will be corrected soon.

After that the test issues some calls to swipeUp(). Note that actionOnItemAtPosition() will scroll the respective item into view so don’t just always pass 0 like I did initially.

A temporary addition to ensure the swipes actually force the creation of additional list items is needed:

// ...
public class WatchlistRecyclerViewAdapter 
        extends RecyclerView.Adapter<WatchlistRecyclerViewAdapter.ViewHolder> {
    // ...

    @Override
    public void onBindViewHolder(final ViewHolder viewHolder, int reportPosition) {
        // ...
        // Symbol
        viewHolder.Symbol = this.cursor.getString(
                this.cursor.getColumnIndex(SecurityContract.Security.SYMBOL));
        Log.d(CLASS_NAME, "onBindViewHolder(): symbol = " + viewHolder.Symbol);
        // SymbolTextView
        viewHolder.SymbolTextView.setText(this.cursor.getString(
                this.cursor.getColumnIndex(SecurityContract.Security.SYMBOL)));
        // ...
    }

    // ...
}

That needs to be commented out after verifying the test works as expected – onBindViewHolder() is the last spot you want to slow down.

Start the test now by clicking Run ‘scrollingExperienceTest()’ in the method’s context menu. You should be able to watch a tap on Refresh followed by the additional reports showing up and swipes to the end of them. Android Studio’s logcat window should show something like this:

...
... D/QuoteRefresherService: onHandleIntent(): quotes updated - initiating screen refresh
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = BAYN.DE
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = SIE.DE
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = ABBN.VX
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = NESN.VX
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = NOVN.VX
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = CFR.VX
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp0
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 0' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = ROG.VX
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp1
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 1' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = SYNN.VX
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp2
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 2' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = UBSG.VX
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp3
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 3' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/WatchlistRec.ViewAd.: onBindViewHolder(): symbol = ZURN.VX
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp4
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 4' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp5
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 5' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp6
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 6' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/ScrollingExperienceTest: scrollingExperienceTest(): swipeUp7
... I/ViewInteraction: Performing 'actionOnItemAtPosition performing ViewAction: fast swipe on item at position: 7' action on view (with id: de.dbremes.dbtradealert.playStore:id/list and has parent matching: (with id: de.dbremes.dbtradealert.playStore:id/container and has parent matching: with id: de.dbremes.dbtradealert.playStore:id/main_content) and is displayed on the screen to the user)
... D/ScrollingExperienceTest: scrollingExperienceTest(): the End
... D/DbHelper: deleteTestSecurities(): success!

So as required the swipes cause RecyclerView items to be created.

Next post: Automated UI Tests – Part 3: Testing with WireMock and Firebase Test Lab for Android

Additional Resources

Advertisements
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s