"Unit test success" in Sonarqube 4.4

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

"Unit test success" in Sonarqube 4.4

Lorenzo Bettini
Hi

in Sonarqube 4.4 I don't see the section "Unit test success" in the
"Unit test coverage" widget...  should I install a specific plugin?

thanks in advance
        Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Nicolas Peru
Hi, 

No need for a specific plugin. This is probably due to the fact that the java plugin (more specifically the surefire sensor) can't map your test classes to test files. I suppose you have a upgraded from a version prior to 4.2 ? You can try to run an analysis with debug output and double check your test source configuration.

Cheers, 

Nicolas PERU | SonarSource
Senior Developer
http://sonarsource.com



On 5 September 2014 23:50, Lorenzo Bettini <[hidden email]> wrote:
Hi

in Sonarqube 4.4 I don't see the section "Unit test success" in the
"Unit test coverage" widget...  should I install a specific plugin?

thanks in advance
        Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Lorenzo Bettini
On 06/09/2014 00:18, Nicolas Peru wrote:
> Hi,
>
> No need for a specific plugin. This is probably due to the fact that the
> java plugin (more specifically the surefire sensor) can't map your test
> classes to test files. I suppose you have a upgraded from a version
> prior to 4.2 ? You can try to run an analysis with debug output and
> double check your test source configuration.
>

Hi

I did some further experiments with sonar 3.7.4 and sonarqube 4.4...

first of all I'm analyzing an eclipse project (with many sub-projects),
with Maven/Tycho, but I'm getting the same results with the maven/tycho
examples which is part of sonar examples:
https://github.com/SonarSource/sonar-examples/tree/master/projects/tycho

whose pom.xml defineds these properties

    <!-- Sonar -->
    <sonar.language>java</sonar.language>
    <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>

<sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>

<sonar.junit.reportsPath>../${project.artifactId}.tests/target/surefire-reports/</sonar.junit.reportsPath>
    <sonar.skippedModules>plugin.tests</sonar.skippedModules>

As you see the plugin.tests is skipped

SONAR 3.7.4

by running the analysis, i.e.,

mvn clean install
mvn sonar:sonar

the relevant line is

[INFO] [15:48:52.315] Sensor SurefireSensor...
[INFO] [15:48:52.316] parsing
/home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
[INFO] [15:48:52.398] Sensor SurefireSensor done: 82 ms

and sonar web shows the 100% tests success
(of course you can't browse the source of the test)

SONARQUBE 4.4

Now, let's switch to sonarqube 4.4 (but I tried also 4.3); same
configuration, same run, but I get (with sonar.verbose set to true)

[INFO] [16:00:58.156] Sensor SurefireSensor...
[INFO] [16:00:58.156] parsing
/home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
[DEBUG] [16:00:58.230] Class not found in SquidIndex: example.GreetingTest
[WARN] [16:00:58.230] Resource not found: example.GreetingTest
[INFO] [16:00:58.230] Sensor SurefireSensor done: 74 ms

Even removing the skip.modules (i.e., examining tests as well), I still
get the above message, and sonar we shows no test result...

this sounds like a bug?

thanks in advance
        Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Lorenzo Bettini
anyone?

On 06/09/2014 16:04, Lorenzo Bettini wrote:

> On 06/09/2014 00:18, Nicolas Peru wrote:
>> Hi,
>>
>> No need for a specific plugin. This is probably due to the fact that the
>> java plugin (more specifically the surefire sensor) can't map your test
>> classes to test files. I suppose you have a upgraded from a version
>> prior to 4.2 ? You can try to run an analysis with debug output and
>> double check your test source configuration.
>>
>
> Hi
>
> I did some further experiments with sonar 3.7.4 and sonarqube 4.4...
>
> first of all I'm analyzing an eclipse project (with many sub-projects),
> with Maven/Tycho, but I'm getting the same results with the maven/tycho
> examples which is part of sonar examples:
> https://github.com/SonarSource/sonar-examples/tree/master/projects/tycho
>
> whose pom.xml defineds these properties
>
>     <!-- Sonar -->
>     <sonar.language>java</sonar.language>
>     <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
>     <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
>
> <sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
>
> <sonar.junit.reportsPath>../${project.artifactId}.tests/target/surefire-reports/</sonar.junit.reportsPath>
>     <sonar.skippedModules>plugin.tests</sonar.skippedModules>
>
> As you see the plugin.tests is skipped
>
> SONAR 3.7.4
>
> by running the analysis, i.e.,
>
> mvn clean install
> mvn sonar:sonar
>
> the relevant line is
>
> [INFO] [15:48:52.315] Sensor SurefireSensor...
> [INFO] [15:48:52.316] parsing
> /home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
> [INFO] [15:48:52.398] Sensor SurefireSensor done: 82 ms
>
> and sonar web shows the 100% tests success
> (of course you can't browse the source of the test)
>
> SONARQUBE 4.4
>
> Now, let's switch to sonarqube 4.4 (but I tried also 4.3); same
> configuration, same run, but I get (with sonar.verbose set to true)
>
> [INFO] [16:00:58.156] Sensor SurefireSensor...
> [INFO] [16:00:58.156] parsing
> /home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
> [DEBUG] [16:00:58.230] Class not found in SquidIndex: example.GreetingTest
> [WARN] [16:00:58.230] Resource not found: example.GreetingTest
> [INFO] [16:00:58.230] Sensor SurefireSensor done: 74 ms
>
> Even removing the skip.modules (i.e., examining tests as well), I still
> get the above message, and sonar we shows no test result...
>
> this sounds like a bug?
>
> thanks in advance
> Lorenzo
>


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Julien HENRY
Hi Lorenzo,

I have updated the Tycho sample project to better works with last versions of SQ/SQ Java. The key point as you know is that in Eclipse world tests are in a dedicated subproject. Because of bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=397015 source folder of a test sub project are reported to be main source code instead of test code. As a result SQ used to analyze this code as main code. That's why we suggested to skip test module from the analysis.
But since SQ Maven plugin 2.4 it is possible to override configuration reported by Maven. See what I did:
https://github.com/SonarSource/sonar-examples/blob/master/projects/tycho/plugin.tests/pom.xml#L17

As a result it is no more necessary to skip test sub project from SQ analysis. Also the hack that used to import surefire report of test subproject into the main subproject should also be removed (this is what caused the display of Resource not found: example.GreetingTest).
https://github.com/SonarSource/sonar-examples/commit/dac73ca970aa274e3ccbcfbfdd5889113b06686d#diff-720fbbe73f8e02938c2a1c0514fa7e90L26

With these changes I have:
  - 100% coverage on main sub project
  - 100% test success and details of test execution on test subproject
According to me this is more accurate that what we had previously (since tests a indeed located in test subproject)

What is missing in a perfect world is coverage per test details but we don't support "cross project test coverage" for now (I mean code in a subproject that is covered by test in another subproject).

++

Julien

2014-09-10 11:37 GMT+02:00 Lorenzo Bettini <[hidden email]>:
anyone?

On 06/09/2014 16:04, Lorenzo Bettini wrote:
> On 06/09/2014 00:18, Nicolas Peru wrote:
>> Hi,
>>
>> No need for a specific plugin. This is probably due to the fact that the
>> java plugin (more specifically the surefire sensor) can't map your test
>> classes to test files. I suppose you have a upgraded from a version
>> prior to 4.2 ? You can try to run an analysis with debug output and
>> double check your test source configuration.
>>
>
> Hi
>
> I did some further experiments with sonar 3.7.4 and sonarqube 4.4...
>
> first of all I'm analyzing an eclipse project (with many sub-projects),
> with Maven/Tycho, but I'm getting the same results with the maven/tycho
> examples which is part of sonar examples:
> https://github.com/SonarSource/sonar-examples/tree/master/projects/tycho
>
> whose pom.xml defineds these properties
>
>     <!-- Sonar -->
>     <sonar.language>java</sonar.language>
>     <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
>     <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
>
> <sonar.jacoco.reportPath>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPath>
>
> <sonar.junit.reportsPath>../${project.artifactId}.tests/target/surefire-reports/</sonar.junit.reportsPath>
>     <sonar.skippedModules>plugin.tests</sonar.skippedModules>
>
> As you see the plugin.tests is skipped
>
> SONAR 3.7.4
>
> by running the analysis, i.e.,
>
> mvn clean install
> mvn sonar:sonar
>
> the relevant line is
>
> [INFO] [15:48:52.315] Sensor SurefireSensor...
> [INFO] [15:48:52.316] parsing
> /home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
> [INFO] [15:48:52.398] Sensor SurefireSensor done: 82 ms
>
> and sonar web shows the 100% tests success
> (of course you can't browse the source of the test)
>
> SONARQUBE 4.4
>
> Now, let's switch to sonarqube 4.4 (but I tried also 4.3); same
> configuration, same run, but I get (with sonar.verbose set to true)
>
> [INFO] [16:00:58.156] Sensor SurefireSensor...
> [INFO] [16:00:58.156] parsing
> /home/bettini/git/sonar-examples/projects/tycho/plugin.tests/target/surefire-reports
> [DEBUG] [16:00:58.230] Class not found in SquidIndex: example.GreetingTest
> [WARN] [16:00:58.230] Resource not found: example.GreetingTest
> [INFO] [16:00:58.230] Sensor SurefireSensor done: 74 ms
>
> Even removing the skip.modules (i.e., examining tests as well), I still
> get the above message, and sonar we shows no test result...
>
> this sounds like a bug?
>
> thanks in advance
>       Lorenzo
>


--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Lorenzo Bettini
On 10/09/2014 13:00, Julien HENRY wrote:

> Hi Lorenzo,
>
> I have updated the Tycho sample project to better works with last
> versions of SQ/SQ Java. The key point as you know is that in Eclipse
> world tests are in a dedicated subproject. Because of bug
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=397015 source folder of a
> test sub project are reported to be main source code instead of test
> code. As a result SQ used to analyze this code as main code. That's why
> we suggested to skip test module from the analysis.
> But since SQ Maven plugin 2.4 it is possible to override configuration
> reported by Maven. See what I did:
> https://github.com/SonarSource/sonar-examples/blob/master/projects/tycho/plugin.tests/pom.xml#L17
>
> As a result it is no more necessary to skip test sub project from SQ
> analysis. Also the hack that used to import surefire report of test
> subproject into the main subproject should also be removed (this is what
> caused the display of Resource not found: example.GreetingTest).
> https://github.com/SonarSource/sonar-examples/commit/dac73ca970aa274e3ccbcfbfdd5889113b06686d#diff-720fbbe73f8e02938c2a1c0514fa7e90L26
>
> With these changes I have:
>   - 100% coverage on main sub project
>   - 100% test success and details of test execution on test subproject
> According to me this is more accurate that what we had previously (since
> tests a indeed located in test subproject)

Hi Julien

thank you for the update, now it works also in my project!

I see this issue though (well... this is not a blocking issue, only
something that is somehow annoying):

- this requires also test projects to be analyzed... of course, this is
considered good practice (also tests must be clean code), but this
requires more time and you may want to concentrate on the real code (and
only now and then analyze tests as well).  This can be dealt with by
using something like

<sonar.exclusions>file:**/tests/**</sonar.exclusions>

however, I noted that test files will be excluded only by the squid
analysis: findbugs will still analyze them.  Is that intentional?  Of
course one can provide an exclude filter for findbugs.

(This also highlighted a strange behavior of findbugs that I'll describe
in another post).

thanks
        Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: "Unit test success" in Sonarqube 4.4

Julien HENRY
Hi,

Test files will indeed be indexed but not analyzed since today SQ Java plugin do not report issues on test files. Concerning findbugs it may be a bit different since it works at bytecode level so I'm not sure the Findbugs plugin take exclusions into account.

++

Julien

2014-09-13 10:02 GMT+02:00 Lorenzo Bettini <[hidden email]>:
On 10/09/2014 13:00, Julien HENRY wrote:
> Hi Lorenzo,
>
> I have updated the Tycho sample project to better works with last
> versions of SQ/SQ Java. The key point as you know is that in Eclipse
> world tests are in a dedicated subproject. Because of bug
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=397015 source folder of a
> test sub project are reported to be main source code instead of test
> code. As a result SQ used to analyze this code as main code. That's why
> we suggested to skip test module from the analysis.
> But since SQ Maven plugin 2.4 it is possible to override configuration
> reported by Maven. See what I did:
> https://github.com/SonarSource/sonar-examples/blob/master/projects/tycho/plugin.tests/pom.xml#L17
>
> As a result it is no more necessary to skip test sub project from SQ
> analysis. Also the hack that used to import surefire report of test
> subproject into the main subproject should also be removed (this is what
> caused the display of Resource not found: example.GreetingTest).
> https://github.com/SonarSource/sonar-examples/commit/dac73ca970aa274e3ccbcfbfdd5889113b06686d#diff-720fbbe73f8e02938c2a1c0514fa7e90L26
>
> With these changes I have:
>   - 100% coverage on main sub project
>   - 100% test success and details of test execution on test subproject
> According to me this is more accurate that what we had previously (since
> tests a indeed located in test subproject)

Hi Julien

thank you for the update, now it works also in my project!

I see this issue though (well... this is not a blocking issue, only
something that is somehow annoying):

- this requires also test projects to be analyzed... of course, this is
considered good practice (also tests must be clean code), but this
requires more time and you may want to concentrate on the real code (and
only now and then analyze tests as well).  This can be dealt with by
using something like

<sonar.exclusions>file:**/tests/**</sonar.exclusions>

however, I noted that test files will be excluded only by the squid
analysis: findbugs will still analyze them.  Is that intentional?  Of
course one can provide an exclude filter for findbugs.

(This also highlighted a strange behavior of findbugs that I'll describe
in another post).

thanks
        Lorenzo

--
Lorenzo Bettini, PhD in Computer Science, DI, Univ. Torino
HOME: http://www.lorenzobettini.it
Xtext Book:
http://www.packtpub.com/implementing-domain-specific-languages-with-xtext-and-xtend/book



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email