Quantcast

Gallio & Row tests

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Gallio & Row tests

Andrew Ochsner
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Fabrice Bellingard-4
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Andrew Ochsner
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.

On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Fabrice Bellingard-4
Can you send us the excerpt of the report where the number of tests appears?

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Andrew Ochsner
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Fabrice Bellingard-4
Actually, the parser that analyses the test results focuses on <testStep> tags, so I was looking for the excerpt containing those tags for the given test method.


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 4:24 PM, Andrew Ochsner <[hidden email]> wrote:
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:<a href="tel:30.673258-06" value="+13067325806" target="_blank">30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Andrew Ochsner
yep, sorry about that!  Here's an example.  Let me know if there's more you need.  Looks like the testId is the same but the name differs w/ the parameters passed in by the Row attribute

<children>
<testStepRun startTime="2012-06-21T22:02:33.8628336-06:00" endTime="2012-06-21T22:02:40.7703254-06:00">
<testStep id="97de4881df59af0d" name="Validate(740, {LowerRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(740, {LowerRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="6.8172330999999993"/>
<testLog/>
</testStepRun>
<testStepRun startTime="2012-06-21T22:02:40.7830212-06:00" endTime="2012-06-21T22:02:40.7937638-06:00">
<testStep id="eb3efb679b9b95e7" name="Validate(760, {LowestRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(760, {LowestRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="0.0004637"/>
<testLog/>
</testStepRun>
</children>

On Fri, Jun 22, 2012 at 11:36 AM, Fabrice Bellingard <[hidden email]> wrote:
Actually, the parser that analyses the test results focuses on <testStep> tags, so I was looking for the excerpt containing those tags for the given test method.


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 4:24 PM, Andrew Ochsner <[hidden email]> wrote:
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:<a href="tel:30.673258-06" value="+13067325806" target="_blank">30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner






Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Fabrice Bellingard-4
OK, I think you can create a JIRA ticket about this (please add a link to this discussion in the description of the ticket => http://sonar.15.n6.nabble.com/Gallio-amp-Row-tests-td5000086.html)

Thanks Andrew!

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 6:48 PM, Andrew Ochsner <[hidden email]> wrote:
yep, sorry about that!  Here's an example.  Let me know if there's more you need.  Looks like the testId is the same but the name differs w/ the parameters passed in by the Row attribute

<children>
<testStepRun startTime="2012-06-21T22:02:33.8628336-06:00" endTime="2012-06-21T22:02:40.7703254-06:00">
<testStep id="97de4881df59af0d" name="Validate(740, {LowerRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(740, {LowerRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="6.8172330999999993"/>
<testLog/>
</testStepRun>
<testStepRun startTime="2012-06-21T22:02:40.7830212-06:00" endTime="2012-06-21T22:02:40.7937638-06:00">
<testStep id="eb3efb679b9b95e7" name="Validate(760, {LowestRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(760, {LowestRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="0.0004637"/>
<testLog/>
</testStepRun>
</children>

On Fri, Jun 22, 2012 at 11:36 AM, Fabrice Bellingard <[hidden email]> wrote:
Actually, the parser that analyses the test results focuses on <testStep> tags, so I was looking for the excerpt containing those tags for the given test method.


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 4:24 PM, Andrew Ochsner <[hidden email]> wrote:
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:<a href="tel:30.673258-06" value="+13067325806" target="_blank">30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner







Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Andrew Ochsner
Thanks.  Created  http://jira.codehaus.org/browse/SONARPLUGINS-1996   

Let me know if there's more I should add.

Andy O

On Mon, Jun 25, 2012 at 3:09 AM, Fabrice Bellingard <[hidden email]> wrote:
OK, I think you can create a JIRA ticket about this (please add a link to this discussion in the description of the ticket => http://sonar.15.n6.nabble.com/Gallio-amp-Row-tests-td5000086.html)

Thanks Andrew!


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 6:48 PM, Andrew Ochsner <[hidden email]> wrote:
yep, sorry about that!  Here's an example.  Let me know if there's more you need.  Looks like the testId is the same but the name differs w/ the parameters passed in by the Row attribute

<children>
<testStepRun startTime="2012-06-21T22:02:33.8628336-06:00" endTime="2012-06-21T22:02:40.7703254-06:00">
<testStep id="97de4881df59af0d" name="Validate(740, {LowerRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(740, {LowerRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="6.8172330999999993"/>
<testLog/>
</testStepRun>
<testStepRun startTime="2012-06-21T22:02:40.7830212-06:00" endTime="2012-06-21T22:02:40.7937638-06:00">
<testStep id="eb3efb679b9b95e7" name="Validate(760, {LowestRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(760, {LowestRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="0.0004637"/>
<testLog/>
</testStepRun>
</children>

On Fri, Jun 22, 2012 at 11:36 AM, Fabrice Bellingard <[hidden email]> wrote:
Actually, the parser that analyses the test results focuses on <testStep> tags, so I was looking for the excerpt containing those tags for the given test method.


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 4:24 PM, Andrew Ochsner <[hidden email]> wrote:
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:<a href="tel:30.673258-06" value="+13067325806" target="_blank">30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner








Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Gallio & Row tests

Fabrice Bellingard-4
That's perfect, thanks!

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Mon, Jun 25, 2012 at 4:07 PM, Andrew Ochsner <[hidden email]> wrote:
Thanks.  Created  http://jira.codehaus.org/browse/SONARPLUGINS-1996   

Let me know if there's more I should add.

Andy O


On Mon, Jun 25, 2012 at 3:09 AM, Fabrice Bellingard <[hidden email]> wrote:
OK, I think you can create a JIRA ticket about this (please add a link to this discussion in the description of the ticket => http://sonar.15.n6.nabble.com/Gallio-amp-Row-tests-td5000086.html)

Thanks Andrew!


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 6:48 PM, Andrew Ochsner <[hidden email]> wrote:
yep, sorry about that!  Here's an example.  Let me know if there's more you need.  Looks like the testId is the same but the name differs w/ the parameters passed in by the Row attribute

<children>
<testStepRun startTime="2012-06-21T22:02:33.8628336-06:00" endTime="2012-06-21T22:02:40.7703254-06:00">
<testStep id="97de4881df59af0d" name="Validate(740, {LowerRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(740, {LowerRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="6.8172330999999993"/>
<testLog/>
</testStepRun>
<testStepRun startTime="2012-06-21T22:02:40.7830212-06:00" endTime="2012-06-21T22:02:40.7937638-06:00">
<testStep id="eb3efb679b9b95e7" name="Validate(760, {LowestRiskScore})" fullName="Common.UnitTest/CreditRiskProfileScoreTest/Validate/Validate(760, {LowestRiskScore})" testId="66b53f53a7a2d9bd" isPrimary="false" isTestCase="true" isDynamic="false">
<codeReference assembly="Common.UnitTest, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" namespace="Common.UnitTest" type="Common.UnitTest.CreditRiskProfileScoreTest" member="Validate"/>
<codeLocation path="d:\Common\UnitTest\CreditRiskProfileScoreTest.cs" line="46"/>
<metadata>
<entry key="TestKind">
<value>Test</value>
</entry>
</metadata>
</testStep>
<children/>
<result assertCount="1" duration="0.0004637"/>
<testLog/>
</testStepRun>
</children>

On Fri, Jun 22, 2012 at 11:36 AM, Fabrice Bellingard <[hidden email]> wrote:
Actually, the parser that analyses the test results focuses on <testStep> tags, so I was looking for the excerpt containing those tags for the given test method.


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 4:24 PM, Andrew Ochsner <[hidden email]> wrote:
<report>
<testPackage shadowCopy="false" debug="false"/>
<testModel>
<test id="c2722843734a7a54" name="Root" fullName="" isTestCase="false"/>
<annotations/>
</testModel>
<testPackageRun startTime="2012-06-21T22:02:<a href="tel:30.673258-06" value="+13067325806" target="_blank">30.673258-06:00" endTime="2012-06-21T22:03:22.0365584-06:00">
<testStepRun startTime="2012-06-21T22:02:33.510281-06:00" endTime="2012-06-21T22:03:22.0385116-06:00"/>
<statistics assertCount="772" duration="51.3697198" runCount="78" passedCount="78" failedCount="0" inconclusiveCount="0" skippedCount="17" testCount="95" stepCount="136">
<outcomeSummaries>
<outcomeSummary count="78">
<outcome status="passed"/>
</outcomeSummary>
<outcomeSummary count="16">
<outcome status="skipped" category="ignored"/>
</outcomeSummary>
<outcomeSummary count="1">
<outcome status="skipped"/>
</outcomeSummary>
</outcomeSummaries>
</statistics>
</testPackageRun>
<logEntries/>
</report>

On Fri, Jun 22, 2012 at 9:01 AM, Fabrice Bellingard <[hidden email]> wrote:
Can you send us the excerpt of the report where the number of tests appears?


Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Fri, Jun 22, 2012 at 3:48 PM, Andrew Ochsner <[hidden email]> wrote:
We are using OpenCover.  We run the tests/coverage outside of Sonar (sonar.gallio.mode=reuseReport).  The gallio xml report reports the correct number of tests.


On Fri, Jun 22, 2012 at 6:19 AM, Fabrice Bellingard <[hidden email]> wrote:
Hi Andrew,

which coverage engine have you configured for your analysis? There are chances that the problem comes from the rule engine and not from Sonar (we should have a look a the coverage result file to know more about this).

Best regards,

Fabrice BELLINGARD | SonarSource
http://sonarsource.com



On Thu, Jun 21, 2012 at 7:03 PM, Andrew Ochsner <[hidden email]> wrote:
Hello:

Seems the test count that comes back doesn't factor in the Row attribute on tests.  So if I have a test:

        [Test]
        [Row(1, 1, 2) ] 
        [Row(-1, 1, 0) ] 
        [Row(-1, -1, -2) ] 
        public void AdditionTest(int a, int b, int c)
        { ... }

It Sonar only reports 1 test, but it's actually 3 tests with different data for each.  

Any chance I'm missing something or there's a workaround for this?

Thanks
Andy Ochsner









Loading...