Crypto Core - A *Swiss Army knife* for Blockchain related Java projects

Crypto Core has the ambitious target to provide a Swiss Army knife for Blockchain related projects.

<p dir="auto"><center><br /> <img src="https://images.hive.blog/768x0/https://imgur.com/oFCGrOb.png" alt="CryptoCoreLogo" srcset="https://images.hive.blog/768x0/https://imgur.com/oFCGrOb.png 1x, https://images.hive.blog/1536x0/https://imgur.com/oFCGrOb.png 2x" /><br /> <p dir="auto">This project has the ambitious target to provide a <em>Swiss Army knife for Blockchain related projects. <h1>Crypto Core - A <em>Swiss Army knife for Blockchain related Java projects <p dir="auto"><br />Hello Steemians! <p dir="auto">I am really proud to share my latest project with you: It is called <em>Crypto Core and I already had it in mind for a long long time, but never managed to push it until now. <p dir="auto">Everything began with the development of version 0.2.0 of <a href="https://steemit.com/utopian-io/@dez1337/steemj-v0-4-3-is-available-now-use-the-steem-api-in-your-java-project" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">SteemJ. This version added 'Transaction Signing' to the project and the main part of this feature is based on some <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj methods. At this point I was already disappointed of using the whole bitcoinj project with all its dependencies, while only being able to use 1% of their code. <p dir="auto">This disappointed me even more when I've started to develop <a href="https://steemit.com/utopian-io/@dez1337/steemj-image-upload-has-been-released-upload-images-to-steemitimages-com-out-of-your-java-application" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">SteemJ-Image-Upload. This project now has a huge amount of dependencies due to bitcoinj and is not the small, additional lib it was supposed to be. <p dir="auto">So because of that, I've asked the bitcoinj developer to separate the non-bitcoin related code to a separate package in <a href="https://github.com/bitcoinj/bitcoinj/issues/1486" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Issue #1486. Their feedback was quite positive, but so far, I received no feedback in <a href="https://groups.google.com/forum/#!topic/bitcoinj/OgoHv6AgvmA" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">their forum and decided to start something on my own. <p dir="auto">The upcoming chapters will explain the background behind the project and the problem it tries to solve in detail. The content below is based on the README of the project which has been updated in the following commit: <blockquote> <p dir="auto"><span><a href="https://github.com/marvin-we/crypto-core/commit/8362b88fe2d71cfbaf9a9722e574e65f0830a44c" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">https://github.com/marvin-we/crypto-core/commit/8362b88fe2d71cfbaf9a9722e574e65f0830a44c <h1>Background <p dir="auto">It is well known that every Blockchain technology should have its own special features which makes it more or less unique. Because of this, every specific Blockchain requires its own specific library which implements and supports those features. When having a look at <em>Java Api implementations, <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj, <a href="https://github.com/ethereum/ethereumj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">etheriumj or <a href="https://github.com/marvin-we/steem-java-api-wrapper" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">steemj are good examples of those unique libraries. <p dir="auto">However, a lot of Blockchain technologies use the same core mechanisms which every Blockchain related project has to solve. To make this more clear, here are some examples: <ul> <li>SHA256 hashing <li>Base58 encoding/decoding <li>HEX encoding/decoding <li>Handling of private and public keys <li>Signing of messages <li>Handling of common C types like <code>VarInt that do not exist in Java <li>... <p dir="auto">The team around <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj, which is the oldest and most proven <em>Java Api implementation for a Blockchain, already solved all those issue. Sadly, the implementations are specific for Bitcoin and a big part can't be used out of the box for other Blockchain technologies. This fact forced other projects to: <ol> <li>Implement a custom solution (e.g. the IOTA Java Api). <li>Copy the source code of <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj into their own project and adjust it (e.g. etheriumj). <li>Use the full <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj project and build the required changes around it (e.g. steemj). <p dir="auto">The first approach is something you should only take into concideration if you have a really strong crypto and blockchain background as the those topics require a lot of knowledge. <p dir="auto">The second approach is a better idea, as the well tested, documented and proven classes of the <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj project are used. The disadvantage of this approach is that updates and fixes require a manual merge. <p dir="auto">The third approach solves also the problem of manual merges, but requires to have add the whole <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj project with all its dependencies and classes while only using ~2% of their code. <h1>How crypto core solves this problem? <p dir="auto">The idea behind <em>crypto core is to provide a flexible and well documented <em>Swiss Army knife for as much Blockchain technologies as possible. This is archieved by extracting non Bitcoin related classes from the great <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj library and making those methods more flexible. <h2>Example <p dir="auto">The following snippets show the difference between <em>bitcoinj and <em>crypto core using the <code>signMessage as an example. <h3>Using BitcoinJ <p dir="auto">The <code>ECKey object of the <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj project is used in a lot of other Blockchain projects like steemj, etheriumj or graphenej. It provides a <code>signMessage method which can be used to sign a message and, for sure, this is something every Blockchain related project has to do. <p dir="auto">Sadly, the bitcoinj implementation is doing this by creating an SHA256 Hash of the message as you can see in line 2: <pre><code> public String signMessage(String message, @Nullable KeyParameter aesKey) throws KeyCrypterException { byte[] data = Utils.formatMessageForSigning(message); Sha256Hash hash = Sha256Hash.twiceOf(data); ECDSASignature sig = sign(hash, aesKey); // Now we have to work backwards to figure out the recId needed to recover the signature. int recId = -1; for (int i = 0; i < 4; i++) { ECKey k = ECKey.recoverFromSignature(i, sig, hash, isCompressed()); if (k != null && k.pub.equals(pub)) { recId = i; break; } } if (recId == -1) throw new RuntimeException("Could not construct a recoverable key. This should never happen."); int headerByte = recId + 27 + (isCompressed() ? 4 : 0); byte[] sigData = new byte[65]; // 1 header + 32 bytes for R + 32 bytes for S sigData[0] = (byte)headerByte; System.arraycopy(Utils.bigIntegerToBytes(sig.r, 32), 0, sigData, 1, 32); System.arraycopy(Utils.bigIntegerToBytes(sig.s, 32), 0, sigData, 33, 32); return new String(Base64.encode(sigData), Charset.forName("UTF-8")); } <p dir="auto">So only because of the first two lines the whole rest of the method can't be used for other blockchain projects that do not generate the hash of a message in the exact same way. This forces developers to search for other solutions instead of being able to reuse the well tested <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj code. <h3>Using Crypto Core <p dir="auto"><em>Crypto Core solves this issue by removing Bitcoin related functionallities from those methods. In this concret example it allows to provide the hash itself as a parameter so that the remaining 90% of the method can be reused. <pre><code> public String signMessage(Sha256Hash messageHash, @Nullable KeyParameter aesKey) { ECDSASignature sig = sign(messageHash, aesKey); // Now we have to work backwards to figure out the recId needed to // recover the signature. int recId = -1; for (int i = 0; i < 4; i++) { ECKey k = ECKey.recoverFromSignature(i, sig, messageHash, isCompressed()); if (k != null && k.pub.equals(pub)) { recId = i; break; } } if (recId == -1) throw new RuntimeException("Could not construct a recoverable key. This should never happen."); int headerByte = recId + 27 + (isCompressed() ? 4 : 0); byte[] sigData = new byte[65]; // 1 header + 32 bytes for R + 32 bytes // for S sigData[0] = (byte) headerByte; System.arraycopy(CryptoUtils.bigIntegerToBytes(sig.r, 32), 0, sigData, 1, 32); System.arraycopy(CryptoUtils.bigIntegerToBytes(sig.s, 32), 0, sigData, 33, 32); return new String(Base64.encode(sigData), Charset.forName("UTF-8")); } <p dir="auto">This would, e.g., allow etheriumj to use this method with a keccak hash. <h1>Technologies <ul> <li>Java 7 <li>Maven 3+ - for building the project <li>Google Protocol Buffers - for use with serialization and hardware communications <h1>Full Documentation <p dir="auto">As <em>Crypto Core only adopts the method signatures in most cases, the <a href="https://bitcoinj.github.io" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj documentation should still be the best way of searching for something. Beside that <em>Crypto Core profits and extends the original JavaDoc developed by the <a href="https://github.com/bitcoinj/bitcoinj" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">bitcoinj team. <h1>Communication <p dir="auto">If you want to contact me you can: <ul> <li>Use <a href="https://groups.google.com/forum/#!topic/bitcoinj/OgoHv6AgvmA" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">this thread in the bitcoinj google group where I've announced this idea. <li>contact me directly at the <a href="https://discord.gg/fsJjr3Q" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Discord Java Channel. <li>create an <a href="https://github.com/marvin-we/crypto-core/issues" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Issue here at GitHub. <h1>Contribution <p dir="auto">This project is really young and requires the help from every user to improve and become more usable. So every kind of feedback is more than welcome. <p dir="auto">If you want to contribute simply clone the repository: <blockquote> <p dir="auto"><span>git clone <a href="https://github.com/marvin-we/crypto-core.git" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">https://github.com/marvin-we/crypto-core.git <p dir="auto">And submit a pull request :) <h1>Binaries <p dir="auto"><em>Crypto Core binaries are pushed into the maven central repository and can be integrated with a bunch of build management tools like Maven. <h2>Maven <p dir="auto">File: <i>pom.xml <pre><code><dependency> <groupId>eu.bittrade.libs</groupId> <artifactId>crypto-core</artifactId> <version>0.1.0pre1</version> </dependency> <h1>How to build the project <p dir="auto">The project requires Maven and Java to be installed on your machine. It can be build with the default maven command: <blockquote> <p dir="auto">mvn clean package <p dir="auto">The resulting JAR can be found in the target directory as usual. <h1>Bugs and Feedback <p dir="auto">For bugs or feature requests please create a <a href="https://github.com/marvin-we/steem-java-api-wrapper/issues" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">GitHub Issue. <p dir="auto">For general discussions or questions you can also: <ul> <li>Post your questions in the <a href="https://discord.gg/fsJjr3Q" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Discord Java Channel <li>Reply to one of the <em>Crypto Core update posts on <a href="https://steemit.com/@dez1337" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Steemit.com <li>Contact me on <a href="https://steemit.chat/channel/dev" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">steemit.chat <h2>Get in touch! <p dir="auto">Most of my projects are pretty time consuming and I always try to provide some useful stuff to the community. What keeps me going for that is your feedback and your support. For that reason I would love to get some Feedback from you <3. Just contact me here on <a href="https://steemit.com/@dez1337" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Steemit or ping me on <a href="https://github.com/marvin-we/steem-java-api-wrapper" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">GitHub. <hr /> <p dir="auto"><center>If you want to stay up to date or just like the stuff I am doing it would be great if you could upvote and resteem this post =). <p dir="auto">Thanks for reading and best regards,<br /><br /> <span> <a href="/@dez1337">@dez1337 <p dir="auto"><br /><hr /><em>Posted on <a href="https://utopian.io/utopian-io/@dez1337/crypto-core-a-swiss-army-knife-for-blockchain-related-java-projects" target="_blank" rel="noreferrer noopener" title="This link will take you away from hive.blog" class="external_link">Utopian.io - Rewarding Open Source Contributors<hr /> <p>