Thread local storage in Java

One of the rarely known features among developers is Thread-local storage.  The idea is simple and need for it comes in  scenarios where we need data that is … well local for the thread. If we have two threads we that refer to the same global variable but we wanna them to have separate value independently initialized of each other.

Most major programming languages have implementation of the concept. For example C++11 has even the thread_local keyword, Ruby has chosen an API approach .

Java has also an implementation of the concept with  java.lang.ThreadLocal<T> and its subclass java.lang.InheritableThreadLocal<T> since version 1.2, so nothing new and shiny here.
Let’s say that for some reason we need to have an Long specific for our thread. Using Thread local that would simple be

public class ThreadLocalExample {

  public static class SomethingToRun implements Runnable {

    private ThreadLocal threadLocal = new ThreadLocal();

    @Override
    public void run() {
      System.out.println(Thread.currentThread().getName() + " " + threadLocal.get());

      try {
        Thread.sleep(2000);
      } catch (InterruptedException e) {
      }

      threadLocal.set(System.nanoTime());
      System.out.println(Thread.currentThread().getName() + " " + threadLocal.get());
    }
  }


  public static void main(String[] args) {
    SomethingToRun sharedRunnableInstance = new SomethingToRun();

    Thread thread1 = new Thread(sharedRunnableInstance);
    Thread thread2 = new Thread(sharedRunnableInstance);

    thread1.start();
    thread2.start();
  }

}
One possible sample run of the following code will result into :


Thread-0 null

Thread-0 132466384576241

Thread-1 null

Thread-1 132466394296347
At the beginning the value is set to null to both threads, obviously each of them works with separate values since after setting the value to System.nanoTime() on Thread-0 it will not have any effect on the value of Thread-1 exactly as we wanted, a thread scoped long variable.

One nice side effect is a case where the thread calls multiple methods from various classes. They will all be able to use the same thread scoped variable without major API changes. Since the value is not explicitly passed through one might argue it difficult to test and bad for design, but that is a separate topic altogether.

In what areas are popular frameworks using Thread Locals?

Spring being one of the most popular frameworks in Java uses ThreadLocals internally for many parts, easily shown by a simple github search. Most of the usages are related to the current’s user’s actions or information. This is actually one of the main uses for ThreadLocals in JavaEE world, storing information for the current request like in RequestContextHolder :


private static final ThreadLocal requestAttributesHolder = 
    new NamedThreadLocal<RequestAttributes>("Request attributes");
Or the current JDBC connection user credentials in UserCredentialsDataSourceAdapter.

If we get back on RequestContextHolder we can use this class to access all of the current request information for anywhere in our code.
Common use case for this is  LocaleContextHolder that helps us store the current user’s locale.
Mockito uses it to store the current “global” configuration and if we take a look at any framework out there there is a high chance we’ll find it as well.

Thread Locals and Memory Leaks

We learned this awesome little feature so let’s use it all over the place. We can do that but few google searches and we can find out that most out there say ThreadLocal is evil. That’s not exactly true, it is a nice utility but in some contexts it might be easy to create a memory leak.

“Can you cause unintended object retention with thread locals? Sure you can. But you can do this with arrays too. That doesn’t mean that thread locals (or arrays) are bad things. Merely that you have to use them with some care. The use of thread pools demands extreme care. Sloppy use of thread pools in combination with sloppy use of thread locals can cause unintended object retention, as has been noted in many places. But placing the blame on thread locals is unwarranted.” – Joshua Bloch

It is very easy to create a memory leak in your server code using ThreadLocal if it runs on an application server. ThreadLocal context is associated to the thread where it runs, and will be garbaged once the thread is dead. Modern app servers use pool of threads instead of creating new ones on each request meaning you can end up holding large objects indefinitely in your application.  Since the thread pool is from the app server our memory leak could remain even after we unload our application. The fix for this is simple, free up resources you do not need.

One other ThreadLocal misuse is API design. Often I have seen use of RequestContextHolder(that holds ThreadLocal) all over the place, like the DAO layer for example. Later on if one were to call the same DAO methods outside a request like and scheduler for example he would get a very bad surprise.
This create black magic and many maintenance developers who will eventually figure out where you live and pay you a visit. Even though the variables in ThreadLocal are local to the thread they are very much global in your code. Make sure you really need this thread scope before you use it.

More info on the topic

http://en.wikipedia.org/wiki/Thread-local_storage
http://www.appneta.com/blog/introduction-to-javas-threadlocal-storage/
https://plumbr.eu/blog/how-to-shoot-yourself-in-foot-with-threadlocals
http://stackoverflow.com/questions/817856/when-and-how-should-i-use-a-threadlocal-variable
https://plumbr.eu/blog/when-and-how-to-use-a-threadlocal
https://weblogs.java.net/blog/jjviana/archive/2010/06/09/dealing-glassfish-301-memory-leak-or-threadlocal-thread-pool-bad-ide
https://software.intel.com/en-us/articles/use-thread-local-storage-to-reduce-synchronization

This post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

Run, JUnit! Run!!!

JUnit together with JavaScript and SVN are some of the technologies that programmers often start using without even reading a single blog post let alone a book.  Maybe this is a good thing since they look simple enough and understandable so we can use them right away without any manuals, but this also means that they are also underused. In this article we will go through some features of JUnit that I consider very useful.

Parameterized tests 

Sometimes we need to run the same method or functionality with many different inputs and different expected results. One way to do this would be to create separate tests for each of the cases, or you

can use loop but that it would be harder to track down the origin of a possible test failure.

For example if we have the following value object representing rational numbers:


public class RationalNumber {

private final long numerator;
private final long denominator;

public RationalNumber(long numerator, long denominator) {
this.numerator = numerator;
this.denominator = denominator;
}

public long getNumerator() {
return numerator;
}

public long getDenominator() {
return denominator;
}

@Override
public String toString() {
return String.format("%d/%d", numerator, denominator);
}
}

And we have a service class called App with a method convert that divides the number to a rounded value of 5 decimal :

public class App {

/**
* THE Logic
*
* @param number some rational number
* @return BigDecimal rounded to 5 decimal points
*/
public static BigDecimal convert(RationalNumber number) {
BigDecimal numerator = new BigDecimal(number.getNumerator()).
setScale(5, RoundingMode.HALF_UP);

BigDecimal result = numerator.divide(
new BigDecimal(number.getDenominator()),
RoundingMode.HALF_UP);

return result;
}
}

And for the actual AppTest class we have

@RunWith(Parameterized.class)
public class AppTest {

private RationalNumber input;
private BigDecimal expected;

public AppTest(RationalNumber input, BigDecimal expected) {
this.input = input;
this.expected = expected;
}

@Parameterized.Parameters(name = "{index}: number[{0}]= {1}")
public static Collection<Object> data() {
return Arrays.asList(new Object[][]{
{new RationalNumber(1, 2), new BigDecimal("0.50000")},
{new RationalNumber(1, 1), new BigDecimal("1.00000")},
{new RationalNumber(1, 3), new BigDecimal("0.33333")},
{new RationalNumber(1, 5), new BigDecimal("0.20000")},
{new RationalNumber(10000, 3), new BigDecimal("3333.33333")}
});
}

@Test
public void testApp() {
//given the test data
//when
BigDecimal out = App.convert(input);
//then
Assert.assertThat(out, is(equalTo(expected)));
}

}

The Parametrized runner or  @RunWith(Parameterized.class)  enables the “parametrization” or in other words the injection of the collection of values annotated with  @Parameterized.Parameters into the Test constructor where each of the sublist is an parameter list.  This means that each of the RationalNumber objects in the data() method will be injected into the input variable and each of the BigDecimal values would be the expected value, so in our example we have 5 tests.

There is also an optional custom naming of the generated test added in the annotation, so “{index}: number[{0}]= {1}” will be replaced with the appropriate parameters defined in the data() method and the “{index}” placeholder will be the test case index, like in the following image

Running the parametrized tests in IntelliJ Idea 

JUnit rules

The simplest definition of JUnit rules is that they are in a sense an interceptors  and very similar to the Spring aspect oriented programming or Java EE interceptors API. Basically you can do useful things before and after the test execution.
OK so let’s start with some of the built in test rules. One of them is ExternalResource  where the idea is that we setup an external resource and after the teardown garteet the resource was freed up. A classic example of such test is a creation of file, so for that purpose we have a built in class TemporaryFolder but we can also create our own ones for other resources :


public class TheRuleTest {
@Rule
public TemporaryFolder folder = new TemporaryFolder();

@Test
public void someTest() throws IOException {
//given
final File tempFile = folder.newFile("thefile.txt");
//when
tempFile.setExecutable(true) ;
//then
assertThat(tempFile.canExecute(), is(true));
}
}

We could have done this in a @Before and @After blocks and use java temp files but it is easy to forget something and leave some of the files unclosed in some of the scenarios where a test fails.

For example there is also a Timeout rule for methods where if the the execution is not finished in given time limit the test will fail with a Timeout exception. For example to limit the running for 20 milliseconds :
   


@Rule
public MethodRule globalTimeout = new Timeout(20);


We can implement our own rules that can do a policy enforcement or various project specific changes. The only thing that needs to be done is for us to implement the TestRule interface.
A simple scenario to explain the behaviour is to add a rule that prints someting before and after test.


import org.junit.rules.TestRule;

import org.junit.runner.Description;
import org.junit.runners.model.Statement;

public class MyTestRule implements TestRule {

public class MyStatement extends Statement {

private final Statement statement;

public MyStatement(Statement statement) {
this.statement = statement;
}

@Override
public void evaluate() throws Throwable {
System.out.println("before statement" );
statement.evaluate();
System.out.println("after statement");
}

}

@Override
public Statement apply(Statement statement,
Description description) {

System.out.println("apply rule");
return new MyStatement(statement);
}

}


So now that we have our rule we can use it in tests, were the tests will just print out different values :


public class SomeTest {

@Rule
public MyTestRule folder = new MyTestRule();

@Test
public void testA() {
System.out.println("A");
}

@Test
public void testB() {
System.out.println("B");
}
}

When we run a test the following output will be created on the  console output :


apply rule
before statement
A
after statement
apply rule
before statement
B
after statement

From the built in one there is one called ExpectedException that can very useful when trying out testing errors. Additionally there is an option to chain the rules that can be useful in many scenarios.

To sum up

If you wanna say that Spock or TestNG or some library build on top of JUnit have more features than JUnit, than that is probably true.
But you know what? We don’t always have those on our class path and chances are that JUnit is there and already used all over the place. Than why not use it’s full potential ?

Useful links

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

Functional Java collections

There is a lot of functional hype these days so I would give a short overview on what is out there at least when it comes to collections in Java. Personally I like standard collections API but i some cases can be awkward and add additional verboseness. This should not be a problem in latter version of Java 8+. There we would probably worry about not creating callback hell but hey there is no silver bullet for most stuff why should there be one for programming?

The Guava Way

Guava project is one of Google’s core libraries where there are plenty of different core language aspects and problems covered. There are utilities and extensions for everyday usage like : collections, primitives, caching, common annotations, string processing, I/O, Math, Reflections and many others. We will only take a look at the Collections goodies so lets see some of them :

    // list
ImmutableList<String> of =
ImmutableList.of("a", "b", "c", "d");
// Same one for map
ImmutableMap<String, String> map =
ImmutableMap.of("key1", "value1", "key2", "value2");
//list of ints
List<Integer> theList = Ints.asList(1, 2, 3, 4, 522, 5, 6);

The Guava Collections are compatible with the JDK collections since they mostly extend or implement the standard classes. There are several cool utilities that are part of the API and have similar names with the ones from java.util.Collections. Basically any programmer who knows the JDK collections should be able to transfer to Guava easily. The ones for List is called Lists, one for Set is Sets, for Map is Maps and so on for the rest. For example:

//create new List
List<someLongName> list = Lists.newArrayList();
//create new LinkedHashMap
Map<someKeyType, SomeValueType> map = Maps.newLinkedHashMap();

//initalize Array List on the spot
List<String> someList = Lists.newArrayList("one", "two", "three");

//set inital size for readability as well as performance
List<Type> exactly100 = Lists.newArrayListWithCapacity(100);
List<Type> approx100 = Lists.newArrayListWithExpectedSize(100);

Methods corresponding to a particular interface are grouped in a very intuitive manner. There are also some extremely good ways of building cache with various of features :

 Cache<Integer, Customer> cache = CacheBuilder.newBuilder()
.weakKeys()
.maximumSize(10000)
.expireAfterWrite(10, TimeUnit.MINUTES)
.build(new CacheLoader<Integer, Customer>() {

@Override
public Customer load(Integer key) throws Exception {

return retreveCustomerForKey(key);
}
});

Since Guava is available in most of the maven repositories its very easy to add it to your build

lambdaj

The idea behind the project is to manipulate collections in a functional and statically typed way. This is achieved in a way the avoids repetitions of simple tasks we usually do with collections. Repetition makes programmers to go with copy/pasting and creates makes them create bugs by doing so. Accessing collections without explicit looping provides way of filtering, sorting, extraction, grouping, transforming, invoking a method on each item or sum up the elements or fields of those element in a collections. Additionally to all these features lambdaj is also a DSL in a way since it adds very cool “sugar” features to the syntax making it more readable in pseudo-english. This is done with static methods so in order to use them we include them directly:

import static ch.lambdaj.Lambda.*;

As it comes to checking and matching lambdaj relies deeply on Hamcrestmatchers. So for example to create a check for an odd integers and then filter a list with that check:

Matcher<Integer> odd = new Predicate<Integer>() {
public boolean apply(Integer item) {
return item % 2 == 1;
}
};
List<Integer> oddNumbers = filter(odd, asList(1, 2, 3, 4, 5));

and as expected the list will return the list [1,3,5]. Lambdaj take a step further with it’s DSL, for example :

List<Beneficiary> beneficiaries = with(transactions)
.retain(having(on(Transaction.class)
.getQunatity(), lessThan(100)))
.extract(on(Transaction.class).getBeneficiary())
.sort(on(Beneficiary.class).getName());

Performance costs

Although the best way to make your application fast is to have the cleanest code as possible there comes a time when you must optimize.In order to do that there is some info provided by the creators on the memory usage and time. Lambdaj has a performance wiki page with code examples. There are also some test done by other programmers where they compare lambdaj with JDK8 for example. There are also some measurements on memory usage of Guava. As for performance of Guava most of it’s functionality is standard JDK classes builders and utilities so the overhead is minimal. At the end of the day it’s up to you to decide how much effect each of these libraries will have on your project and if that is positive. I’m for the idea that almost every project must have Guava on it’s classpath.

Related links summary

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on! Want to write for the blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact Attila Balazs to contribute!