Pex - First Impressions

Apparently the issues I was having with Pex (ClrMonitorFail errors) were caused by its interaction with TypeMock. This doesn't shock me as having used TypeMock for a while I've learned that any time anything odd starts happening disabling TypeMock is a good idea.

So I disabled TypeMock and started playing with Pex, first impression is that it looks great but my second impression was that the IDE integration seemed a little flaky. My IDE actually crashed many times whilst using it but after a while I learned what to click and not click :)

Anyway I thought I'd start writing down the little that I've found out about Pex in case it is in any way useful to anyone.

First Feelings

I did notice a few interesting things when working with Pex, the first is that it seemed like it c lull you into a false sense of security.

Seeing a whole loading of auto-generated tests passing is great but I quickly began to notice that I could modify the code under test in inappropriate ways and my tests weren't failing. This wasn't Pex's fault though, I just hadn't been thorough enough in telling it what to expect and once I applied more PexAssume values and a few more assertions I definitely felt safer.

Fix It Or Allow It

I found that I was getting some useful tests generated even if I was quite vague:

[PexMethod]
[PexUseType(typeof(AuthorizationService))]
public void overall_behavior_correct(IAuthorizationService authorizationService,
    Account source, Account destination, double amountToTransfer)
{
    new AccountTransferService().Transfer(source, destination, amountToTransfer, authorizationService);
}

The tests Pex was generating were to do with null reference exceptions and invariants that the SUT was enforcing. Obviously initially the tests were failing so I had to tell Pex what I expected the SUT to do in each situation. I could do this using either the "Allow It" or "Fix It" options from the "Pex Results" panel.

If I chose "Fix It" then I would tend to get the following:

My IDE would then close down, gah. However if I persevered though it would update the test, for example:

[PexMethod]
[PexUseType(typeof(AuthorizationService))]
public void overall_behavior_correct(IAuthorizationService authorizationService,
   Account source, Account destination, double amountToTransfer)
{
   PexAssume.IsNotNull((object)source, "source");
   PexAssume.IsTrue(source != destination, "source == destination");

   new AccountTransferService().Transfer(source, destination, amountToTransfer, authorizationService);
}

When I then ran Pex for this method again it wouldn't generate a test that passed in null for source or that passed in source and destination as the same objects.

The alternative to "Fix It" is to select "Allow It" it, when I did that Pex would put this attribute in PexAssemblyInfo.cs

[assembly: PexAllowedExceptionFromAssembly(typeof(ArgumentException), "PexPlay")]

This seems a bit brute force though, I don't want to allow the exception across the entire assembly so I probably need to do a bit more research.

Decimals

One interesting thing that I noticed is that when I setup my methods to take in decimals I got no tests generated, but if I changed the inputs to be doubles I did. Not sure what that's all about but again I need to do some more research.

Mocking

Ignore the layout of this test, its a mess, but I do like the easy way I'm able to specify that I want to use a mock of the authorization service using the PexUseType attribute:

       [PexMethod]
      [PexUseType(typeof(MockAuthorizationService))]
      public void transfers_correctly_between_accounts(IAuthorizationService authorizationService,
          double intialInSource, double initialInDestination, double amountToTransfer)
      {
          PexAssume.IsNotNull(authorizationService);
          PexAssume.IsFalse(amountToTransfer < 0);
          PexAssume.IsTrue(intialInSource > amountToTransfer);

          Account source = new Account(intialInSource);
          Account destination = new Account(initialInDestination);

          double fromSourceBefore = source.Balance;
          double fromDestinationBefore = destination.Balance;

          new AccountTransferService().Transfer(source, destination, amountToTransfer, authorizationService);

          Assert.AreEqual(fromSourceBefore - amountToTransfer, source.Balance);
          Assert.AreEqual(fromDestinationBefore + amountToTransfer, destination.Balance);
      }

The mock service is in this form (see the PDF for more on this but I haven't truly had time to grok it yet):

[PexMock]
public class MockAuthorizationService : IAuthorizationService
{
   public bool AllowTransfer(Account source, Account destination, double amountToTransfer)
   {
       var call = PexOracle.Call(this);
       return call.ChooseResult<bool>();
   }
}

When I run Pex over transfers_correctly_between_accounts it will actually correctly use an instance of MockAuthorizationService and will run a test where that service returns false, causing the transfer to fail as expected. Nice.

Share This - Digg It • Save to del.icio.us • Stumble It! • Kick It • DZone