Garbage collector error

Using 8G of memory is likely the problem as all Java code is memory hungry, so coupled with other code this might lead to swapping.

Check SWAP usage (top command) on the Graylog nodes having this problem.

If these machines are swapping, fixes include...

1) Adding more memory, which is almost surely the best solution.

2) Increasing GC threshold.

3) Moving any other memory hungry code off problem Graylog nodes to other machines.

First you must determine/fix the problem, which is likely SWAP usage or I/O saturation (many heavy I/O processes running on machine).

With how little memory can your application run.   Garbage collection is probably the culprit with a lot of small block to be recovered & merged.
Increasing the memory only postpones the GC to happen until it is a bigger problem. to solve.

So if the system can run in 512 MB, only use 512 MB of memory... And in general code that discards memory in a large rate is probably bad written code.
(unless you know that a run need 10GB to complete without GC, then you can use 10GB of mem). 
I have seen a system written in Java that didn't have a large amount of memory turnover and that had realtime properties.
(It needed a daily restart to keep that way though... Esp. the logging of the system caused a lot of GC starting, During GC the java program is halted.)

Upgrade to a more recent java engine MAY help. No guarantees though. 

I increase  to 12 gig. and i don't running anything on this server (only graylog). still has a:

 Nodes with too long GC pauses (triggered 5 days ago)

There are Graylog nodes on which the garbage collector runs too long. Garbage collection runs should be as short as possible
how to  Increasing GC threshold?