Jump to content
Eternal Lands Official Forums
Sign in to follow this  
alvieboy

Proposal: Unit Testing in the client

Recommended Posts

Hey all,

 

Some of you, who work professionally as programmers, are often required to use (or use at your discretion) some sort of unit testing for your code. I always write some sort of unit tests for my code (even in EL; although I do not commit that code), so to help me not only while developing the code itself, but principally when performing modifications to the code, so that I can run the test battery and ensure to a certain level of confidence, that I did not, by accident, modify (broke) any part of the code.

 

For those who are not accustomed with unit testing, there's is some short intro in Wikipedia you can read.

 

Right now there is no support in EL for unit testing, and that's why I am to propose a simple, non-intrusive, non-mandatory unit testing "framework".

 

The system would work as follows:

 

The will be a separate directory in the CVS (tests/) with its own Makefiles. This allows you to build the client as usual, and not to build/run the tests. In the main Makefile* there would exist a new target (tests) that can be used to build/run the automated tests (either all of them or only some subset, like the ones you're currently working on).

 

I'm just not sure which would be the best existing framework for this (cgreen, check, ...).

 

Anyway, I am using some sort of unit testing right now, if you feel that it could be a nice thing to have then I can add some stuff and as time goes by we can improve the framework and add some more tests.

 

Álvaro

Share this post


Link to post
Share on other sites

I've suggested both functional and unit testing for EL several times in the past. I cannot stress its usefulness in both code quality, thought process and confidence in code.

 

Might it be worth adding a separate tree for unit tests that people can add and use if they want?

Share this post


Link to post
Share on other sites

If this is done, a major effort should go into adding in unit tests for a lot of current functionalit to prevent the regressions that happen regularly. If someone isn't willing to add that, then it will take too long to be very useful and people wont use it, and will keep forgetting to ass to it.

Share this post


Link to post
Share on other sites

Might it be worth adding a separate tree for unit tests that people can add and use if they want?

 

That was the idea. Non-mandatory, but somehow recommended. Same as for documentation.

 

If this is done, a major effort should go into adding in unit tests for a lot of current functionalit to prevent the regressions that happen regularly. If someone isn't willing to add that, then it will take too long to be very useful and people wont use it, and will keep forgetting to ass to it.

 

No.

 

Add unit tests to your code in development. Add unit tests to code you modify (of course, most OpenGL stuff is not "testable", but core functions might be). Still, this is not mandatory. Add them if you feel your development time/maintenance time might shorten if you do write those tests. Otherwise, just don't.

 

Most features/code/functions are so hard to test on their own we shouldn't even bother to do test cases for them. Some others (like linked list manipulation, algorithms, so on) are easily tested outside main code.

 

The idea, and let me put some emphasis on it, is to write tests not only to inprove code quality, but to decrease the development time (testing) and the bug hunting/bug fixing phases.

 

Don't get me wrong on this - the idea is not to take longer to write the code, but shorten it by providing a means of testing isolated stuff without the tedious cycle of "launch, connect, login, issue the feature, crash, debug, and so on).

 

Álvaro

Share this post


Link to post
Share on other sites

Might it be worth adding a separate tree for unit tests that people can add and use if they want?

 

That was the idea. Non-mandatory, but somehow recommended. Same as for documentation.

 

If this is done, a major effort should go into adding in unit tests for a lot of current functionalit to prevent the regressions that happen regularly. If someone isn't willing to add that, then it will take too long to be very useful and people wont use it, and will keep forgetting to ass to it.

 

No.

 

Add unit tests to your code in development. Add unit tests to code you modify (of course, most OpenGL stuff is not "testable", but core functions might be). Still, this is not mandatory. Add them if you feel your development time/maintenance time might shorten if you do write those tests. Otherwise, just don't.

 

Most features/code/functions are so hard to test on their own we shouldn't even bother to do test cases for them. Some others (like linked list manipulation, algorithms, so on) are easily tested outside main code.

 

The idea, and let me put some emphasis on it, is to write tests not only to inprove code quality, but to decrease the development time (testing) and the bug hunting/bug fixing phases.

 

Don't get me wrong on this - the idea is not to take longer to write the code, but shorten it by providing a means of testing isolated stuff without the tedious cycle of "launch, connect, login, issue the feature, crash, debug, and so on).

 

Álvaro

I'm specifically talking about putting unit tests in for current features specifically for regression testing. How many times have we see a new feature break existing functionality or introduce an old bug ... look at issues with #cls as a simple example.

Share this post


Link to post
Share on other sites

I'm with Placid on this one.

 

All I want/wish is to have a framework where I can test my EL code without running EL itself. And that I can run whenever I modify that code.

 

And all I want is to have a framework others (beside myself) can use to run tests against their code, if their code is suitable for them.

 

Applying this to EL as a whole makes absolutely no sense. But for us that do small stuff and are not quite sure it does what it was intended for, having some UT's help a lot.

 

Summarizing:

 

I will add an extra directory (tests/) to el, and everyone is free to put testing code that they feel might help on the development and regression testing,

 

unless, obviously, someone is against.

 

Let's not start a "best coding practices war" here, it was not my intention, and surely it is no ones intention.

 

Cheers to all,

 

Álvaro

Share this post


Link to post
Share on other sites

If they absolutely do not interfere with the 'main' branch, and they are totally separated, I don't see any problem.

But can you give me an example on how that would work?

 

Oh, and let's wait until after the update.

Share this post


Link to post
Share on other sites

On the current system, only one line should be added to Makefile.* :

-include tests/Makefile

The "tests" is a separate target for the build, so unless you do a "make tests", everything goes on as usual.

 

Then in a directory named "tests" we have a Makefile like this one (this is for linux, and uses CUnit):

SUITES=linkedlist chgfile

UNIT_LIBS=-lcunit

CFLAGS+=-I.

SUITE_OBJS=$(foreach SUITE,$(SUITES),test_$(SUITE).o)

tests/suites.c: $(OBJS)
echo "#include <CUnit/CUnit.h>" > tests/suites.c
for s in $(SUITES); do \
	echo "extern int $${s}_init(void);" >> tests/suites.c;\
	echo "extern int $${s}_clean(void);" >> tests/suites.c;\
	echo "extern CU_TestInfo $${s}_tests[];" >> tests/suites.c;\
done

echo "CU_SuiteInfo suites[] = {" >> tests/suites.c
for s in $(SUITES); do \
	echo "{\"$$s\",$${s}_init,$${s}_clean,$${s}_tests}," >> tests/suites.c;\
done
echo  "CU_SUITE_INFO_NULL," >> tests/suites.c

echo "};">>tests/suites.c

.PHONY: tests/suites.c

ELOBJS=$(shell echo "$(OBJS)"| sed s/main.o//)

ALL_TEST_OBJS_I=test_runner.o $(SUITE_OBJS)

ALL_TEST_OBJS=$(foreach OBJECT, $(ALL_TEST_OBJS_I), tests/$(OBJECT))

tests: tests/suites.o tests/test_runner

tests/test_runner: $(ALL_TEST_OBJS) tests/suites.o $(ELOBJS)
$(CC) -o tests/test_runner tests/suites.o $(ALL_TEST_OBJS) $(UNIT_LIBS) $(LDFLAGS) $(ELOBJS)

 

In order to add another test suite, just:

 

* Add your suite name to SUITES= in the tests/Makefile;

* Create your suite, using a helper script (you can also do it by hand). The suite would look like this:

 

#include <CUnit/CUnit.h>
#include "list.h"

int mysuite_init()
{
return 0;
}
int mysuite_clean()
{
return 0;
}

CU_TestInfo mysuite_tests[] =
{
/* { "example test", &example_test }, */
CU_TEST_INFO_NULL
};

 

Just implement your tests there, use the init() and clean() functions if needed, and register your tests in the structure array.

 

Full example of a test suite, with one test case:

#include <CUnit/CUnit.h>
#include "list.h"

static list_node_t *list;

int linkedlist_init()
{
list = NULL;
return 0;
}
int linkedlist_clean()
{
return 0;
}

void add_remove_from_list()
{
char *data = "abcd";
CU_ASSERT( list == NULL );
list_push( &list, data );
CU_ASSERT( list != NULL );
CU_ASSERT( strcmp((char*)list_pop(&list),"abcd")==0 );
CU_ASSERT(list==NULL);
}

CU_TestInfo linkedlist_tests[] =
{
{ "add_remove_from_list", &add_remove_from_list },
CU_TEST_INFO_NULL
};

 

Then just run "make tests". It will create a "tests/test_runner" binary, that will execute the tests.

 

It will output something like this:

$ tests/test_runner 


 CUnit - A Unit testing framework for C - Version 2.1-0
 http://cunit.sourceforge.net/



--Run Summary: Type	  Total	 Ran  Passed  Failed
		   suites		2	   2	 n/a	   0
		   tests		 2	   2	   2	   0
		   asserts	   9	   9	   9	   0

Tests completed with return value 0.

 

Álvaro

Share this post


Link to post
Share on other sites

I even added a memory leak detector, for simple tests.

 

just needs to be called in <>_init() as start_memory_debugger() and in <>_clean as stop_memory_debugger().

 

Right now only shows total amount of memory leaked, but can display who did the allocation:

 

$ tests/test_runner 
 CUnit - A Unit testing framework for C - Version 2.1-0
 http://cunit.sourceforge.net/
******** MEMORY LEAKS DETECTED *******
LOST a total of 12 bytes
TOTAL of 1 leaked pointers

 

This will only probably work in linux, cause it uses some malloc GNU extensions.

 

Álvaro

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×