вторник, 28 мая 2013 г.

2 days of integration of osgi, hadoop and hbase

Spent almost 2 days trying to connect to local hbase from OSGI.
The problem is that hbase client uses hadoop which had this bug when run in containers (osgi, tomcat etc).
Bug tracker guys finally submit a fix to legacy 0.23 branch with fix, so I tried it, but it failed with source/api incompatibility (this issue):
java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socket;)Ljava/io/InputStream;
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:363)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1046)

What I've found in the sources is that - hbase calls this static method
public static java.io.InputStream getInputStream(Socket socket) throws IOException
but 0.23.x branch latest version has this
public static org.apache.hadoop.net.SocketInputWrapper getInputStream(java.net.Socket socket) throws java.io.IOException

Do you see the issue? I don't yet. Normally the second can be used in any place where the fist does... But still it fails either due to OSGI some classloading issue or some implicit hbase/hadoop integration issues, I'm not clear on this yet.

What could I do?
I tried orderly downgrading from 0.23.7 to 0.23.1 (just in case), and fortunately 0.23.1 - worked! It has the same static signiture.

So, if you need to connect to hbase from osgi - you'll have to downgrade required hadoop libs to 0.23.1. Seems there is yet another annoying OSGI-specific bug.

SUMMARY: you can add frameworks and components to your java app just adding their dependencies if you don't have OSGI... because otherwise you'll be dancing with tambourines for days and hacking to get it working!

Комментариев нет:

Отправить комментарий