mirror of https://github.com/vitalif/openscad
240 lines
7.7 KiB
Plaintext
240 lines
7.7 KiB
Plaintext
Running regression tests:
|
|
-------------------------
|
|
|
|
First, install the prerequisite helper programs on your system:
|
|
|
|
<<<<<<< HEAD
|
|
cmake, python, ImageMagick 6.5.9.3 or newer, diff
|
|
|
|
There are binary installer packages of these tools available for Mac,
|
|
Win, Linux, BSD, and other systems. (except maybe diff for Win)
|
|
|
|
Next, get a working qmake GUI build of the main openscad binary working.
|
|
For Windows(TM) it is easiest to get a cross-build working. See
|
|
README.md for how to do this. Lastly, install MCAD under
|
|
openscad/libraries.
|
|
=======
|
|
First, install MCAD.
|
|
|
|
$ cd openscad
|
|
$ git submodule update --init
|
|
>>>>>>> 0d9b20dc143b2acca68496d2ce5e4fd02798289e
|
|
|
|
A) Building test environment
|
|
|
|
Linux, Mac:
|
|
|
|
<<<<<<< HEAD
|
|
$ cd tests
|
|
$ cmake .
|
|
$ make
|
|
|
|
Windows(TM):
|
|
|
|
The Windows tests are cross-compiled from within linux, like so:
|
|
|
|
$ source ./scripts/setenv-mingw-xbuild.sh 64 # (32 for 32-bit system)
|
|
$ ./scripts/release-common.sh mingw64 tests # (mingw32 for 32-bit)
|
|
|
|
Once the package is built, install it into Windows as you would any
|
|
other program. Then start up a command prompt, and run this:
|
|
|
|
cd c:\Program Files\OpenSCAD_Tests\openscad\tests\bin
|
|
c:\python27\python.exe mingw_x_testfile.py
|
|
=======
|
|
Windows + MSVC:
|
|
|
|
The MSVC build hasn't been tested in years. See the README for pointers.
|
|
First, gett the main GUI to build. Then, to build the tests:
|
|
>>>>>>> 0d9b20dc143b2acca68496d2ce5e4fd02798289e
|
|
|
|
This will modify paths in CTestTestfile.cmake for your Win system.
|
|
|
|
Windows + MSVC:
|
|
(This hasn't been tested since circa 2012)
|
|
From the QT command prompt:
|
|
|
|
> cd tests
|
|
> cmake . -DCMAKE_BUILD_TYPE=Release
|
|
> sed -i s/\/MD/\/MT/ CMakeCache.txt
|
|
> cmake .
|
|
> nmake -f Makefile
|
|
|
|
<<<<<<< HEAD
|
|
=======
|
|
Cross compiling Linux->Win32 and testing under Wine:
|
|
|
|
Experimental. Please see openscad/tests/CMingw-cross-env.cmake for instructions
|
|
on attempting to get it to work.
|
|
|
|
>>>>>>> 0d9b20dc143b2acca68496d2ce5e4fd02798289e
|
|
B) Running tests
|
|
|
|
$ ctest Runs tests enabled by default
|
|
$ ctest -R <regex> Runs only matching tests, e.g. ctest -R dxf
|
|
$ ctest -C <configs> Adds extended tests belonging to configs.
|
|
Valid configs:
|
|
Default - Run default tests
|
|
Heavy - Run more time consuming tests (> ~10 seconds)
|
|
Examples - test all examples
|
|
Bugs - test known bugs (tests will fail)
|
|
All - test everything
|
|
|
|
|
|
C) Automatically upload test results (experimental)
|
|
|
|
It's possible to automatically upload tests results to an external
|
|
server. This is good for CI, as well as being able to easily report
|
|
bugs.
|
|
|
|
To enable this feature, add '-DOPENSCAD_UPLOAD_TESTS=1' to the cmake cmd-line, e.g.:
|
|
cmake -DOPENSCAD_UPLOAD_TESTS=1 .
|
|
|
|
Adding a new test:
|
|
------------------
|
|
|
|
1) create a test file at an appropriate location under testdata/
|
|
2) if the test is non-obvious, create a human readable description as comments in the test (or in another file in the same directory in case the file isn't human readable)
|
|
3) if a new test app was written, this must be added to tests/CMakeLists.txt
|
|
4) Add the tests to the test apps for which you want them to run (in tests/CMakeLists.txt)
|
|
5) run the test with the environment variable TEST_GENERATE=1, e.g.:
|
|
$ TEST_GENERATE=1 ctest -R mytest
|
|
(this will generate a mytest-expected.txt file which is used for regression testing)
|
|
6) manually verify that the output is correct (tests/regression/<testapp>/mytest-expected.<suffix>)
|
|
7) run the test normally and verify that it passes:
|
|
$ ctest -R mytest
|
|
|
|
Adding a new example:
|
|
---------------------
|
|
|
|
This is almost the same as adding a new regression test:
|
|
1) Create the example under examples/
|
|
2) run the test with the environment variable TEST_GENERATE=1, e.g.:
|
|
$ TEST_GENERATE=1 ctest -C Examples -R exampleNNN
|
|
(this will generate a exampleNNN-expected.txt file which is used for regression testing)
|
|
3) manually verify that the output is correct (tests/regression/<testapp>/exampleNNN.<suffix>)
|
|
4) run the test normally and verify that it passes:
|
|
$ ctest -C Examples -R exampleNNN
|
|
|
|
Troubleshooting:
|
|
------------------------------
|
|
|
|
0. Headless unix servers
|
|
|
|
If you are attempting to run the tests on a unix-like system but only
|
|
have shell-console access, you may be able to run the tests by using a
|
|
virtual framebuffer program like Xvnc or Xvfb. For example:
|
|
|
|
$ Xvfb :5 -screen 0 800x600x24 &
|
|
$ DISPLAY=:5 ctest
|
|
|
|
or
|
|
|
|
$ xvfb-run ctest
|
|
|
|
Some versions of Xvfb may fail, however.
|
|
|
|
1. Trouble finding libraries on unix
|
|
|
|
To help CMAKE find eigen, OpenCSG, CGAL, Boost, and GLEW, you can use
|
|
environment variables, just like for the main qmake & openscad.pro. Examples:
|
|
|
|
OPENSCAD_LIBRARIES=$HOME cmake .
|
|
CGALDIR=$HOME/CGAL-3.9 BOOSTDIR=$HOME/boost-1.47.0 cmake .
|
|
|
|
Valid variables are as follows:
|
|
|
|
BOOSTDIR, CGALDIR, EIGENDIR, GLEWDIR, OPENCSGDIR, OPENSCAD_LIBRARIES
|
|
|
|
When running, this might help find your locally built libraries (assuming
|
|
you installed into $HOME)
|
|
|
|
Linux: export LD_LIBRARY_PATH=$HOME/lib:$HOME/lib64
|
|
Mac: export DYLD_LIBRARY_PATH=$HOME/lib
|
|
|
|
2. Location of logs
|
|
|
|
Logs of test runs are found in tests/build/Testing/Temporary
|
|
A pretty-printed index.html is in a subdir of tests/build/Testing/Temporary
|
|
Expected results are found in tests/regression/*
|
|
Actual results are found in tests/build/testname-output/*
|
|
|
|
3. Image-based tests takes a long time, they fail, and the log says 'return -11'
|
|
|
|
Imagemagick may have crashed while comparing the expected images to the
|
|
test-run generated (actual) images. You can try using the alternate
|
|
ImageMagick comparison method by by erasing CMakeCache, and re-running
|
|
cmake with -DCOMPARATOR=ncc. This will enable the Normalized Cross
|
|
Comparison method which is less accurate but won't usually crash.
|
|
|
|
4. Testing images fails with 'morphology not found" for ImageMagick in the log
|
|
|
|
Your version of imagemagick is old. Upgrade, or pass -DCOMPARATOR=old to
|
|
cmake. The comparison will be of lowered reliability.
|
|
|
|
5. Locale errors
|
|
|
|
"terminate called after throwing an instance of 'std::runtime_error'
|
|
what(): locale::facet::_S_create_c_locale name not valid"
|
|
|
|
Is a boost/libstdc++ bug. Fix like so before running:
|
|
|
|
$ export LC_MESSAGES=
|
|
|
|
<<<<<<< HEAD
|
|
6. Proprietary GL driver issues
|
|
=======
|
|
6. I want to build without OpenGL
|
|
|
|
There is an unsupported way to do this, by defining NULLGL to Cmake:
|
|
|
|
mkdir nullglbin
|
|
cd nullglbin && cmake .. -DNULLGL=1 && make
|
|
|
|
The resulting openscad_nogui binary will fail most tests, but may be
|
|
useful for debugging and outputting 3d-formats like STL on systems without GL.
|
|
This option may break in the future and require tweaking to get working again.
|
|
|
|
7. Other issues
|
|
>>>>>>> 0d9b20dc143b2acca68496d2ce5e4fd02798289e
|
|
|
|
There are sporadic reports of problems running on remote machines with
|
|
proprietary GL drivers. Try doing a web search for your exact error
|
|
message to see solutions others have found.
|
|
|
|
7. Other issues
|
|
|
|
The OpenSCAD User Manual Wiki has a section on buildling. Please check
|
|
there for possible updates and workarounds:
|
|
|
|
http://en.wikibooks.org/wiki/OpenSCAD_User_Manual
|
|
|
|
<<<<<<< HEAD
|
|
Please report build errors (after double checking the instructions) in
|
|
the github issue tracker
|
|
|
|
http://github.com/openscad/openscad/issues
|
|
=======
|
|
|
|
|
|
Migration away from dedicated regression tests:
|
|
-----------------------------------------------
|
|
|
|
In 2013 the test programs underwent a major change. These notes are leftover.
|
|
|
|
This test still needs an intermediate script that mangles away timestamps and
|
|
near-zero floating point numbers:
|
|
|
|
* cgalstlsanitytest
|
|
|
|
Some tests are yet to be converted:
|
|
|
|
* csgtexttest -- verify whether this is not redundant with dumptest
|
|
|
|
These look like tests, but are not actually in use:
|
|
|
|
* modulecachetest
|
|
* cgalcachetest
|
|
>>>>>>> 0d9b20dc143b2acca68496d2ce5e4fd02798289e
|
|
|