Jump to content
Sign in to follow this  
DarkClaw

Is wallet recognizing all the blocks?

Recommended Posts

I'm trying to build a basic block explorer tool (have not done this before) and searched blk0000.dat for the "magic bytes" (apparently cc be b4 fe). I also went through and extracted each block by indexing from the "size bytes" (second four of each block) and parsed the date from the header then counted the number of valid dates.

At 2018-07-15 21:00:15 UTC, both the electron wallet and ccminer report being on block 6734, but I count 6762 blocks using both methods above. So that is 28 "extra" blocks, and I believe that has increased from 25 earlier today (when I first noticed this).

Here are the first 6 dates:

2018-07-13 08:31:53 
2018-07-13 11:55:39 
2018-07-13 11:56:50 
2018-07-13 12:01:52 
2018-07-13 12:04:19 
2018-07-13 12:12:40

And last 6 dates:

2018-07-15 20:56:44 
2018-07-15 20:57:07 
2018-07-15 20:57:14 
2018-07-15 20:57:55 
2018-07-15 20:58:29
2018-07-15 20:59:17

Normally I would assume the bug is mine but I got my count via two different methods and this project is only days new...

Edited by DarkClaw
typo

Share this post


Link to post
Share on other sites

Thinking more on it, I may be seeing only the neoscrpyt blocks in ccminer. Is the wallet not reporting the sha256 blocks?

Edited by DarkClaw
clarify

Share this post


Link to post
Share on other sites

Now by finding duplicates in the hashes of the previous blocks I see that the majority of these "extras" found in the blk*.dat file are orphaned blocks. However, at the moment I count 32 orphans but have an excess of 33 blocks... anyway it looks like this is probably crypto-dev 101 issues.

Share this post


Link to post
Share on other sites

Yes indeed, the blk* files also contain orphan blocks (you can see those in the daemon through "getchaintips" if you want).  Instead of parsing the blk* files, you can also build your block explorer's database through the RPC interface of the daemon - have you considered that?  This way, there's a bit less things to worry about.  You can use "getblockhash" and "getblock" to read blocks, and you can look into the ZMQ interface for staying up-to-date about new blocks (and reorgs) once you are synced initially.

(But note that I've never written a block explorer, so I can't really tell you how others do it.)

Share this post


Link to post
Share on other sites

 

Quote

  Instead of parsing the blk* files, you can also build your block explorer's database through the RPC interface of the daemon - have you considered that?

First, really I am trying to do this as a learning project more than anything. Somehow I am more motivated if I can start with a "fresh" chain than with btc, even though obviously much more documentation is available for that one. So I will probably try doing this both via the available interfaces and "manually" to see how far I can get. Will the rpc interface work on windows though? I tried to follow the instructions I found for btc but it said not supported at the first step:

Quote

To use locally, first start the program in daemon mode:

bitcoind -daemon

https://en.bitcoinwiki.org/wiki/Bitcoind

F:\xaya\ElectronWallet\XAYA Electron\resources\daemon>xayad -daemon
Error: -daemon is not supported on this operating system

 

Also, is there any guide to parsing the blocks available? Currently I've been working off whats available for bitcoin:

https://bitcoin.org/en/developer-reference#raw-transaction-format

https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer

According to that the first byte after the header is the number of transactions. This seems to be different for xaya since that byte is only either 02 or 81 in about equal proportions. Is that a flag for the mining algo or something?

Eg, I've got this for the genesis block:
 

$block$magic
[1] cc be b4 fe

$block$size
[1] 4b 01 00 00

$block$header
 [1] 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[37] fd f8 46 6d 1a 52 b6 8a c5 a2 f3 ef 2e 0e eb fa 40 f0 ba 78 0a f2 3c 8c 97 43 ab 75 1b 90 27 08 79 63 48 5b
[73] 00 00 00 00 00 00 00 00

$block$data
  [1] 02 f0 ff 0f 1e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 [37] 00 00 00 00 00 51 46 4c 7a c4 7e 70 ec 5c 0a d7 6f 62 f2 de a8 63 0b 92 c9 6a 82 93 f4 42 0c f5 e5 76 2d 06
 [73] e5 00 00 00 00 00 00 00 00 27 5b 07 00 01 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[109] 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff ff ff 52 4c 50 48 55 43 20 23 32 2c 33 35 31 2c 38 30 30
[145] 3a 20 38 37 33 30 65 61 36 35 30 64 32 34 63 64 30 31 36 39 32 61 35 61 64 62 39 34 33 65 37 62 38 37 32 30
[181] 62 30 62 61 38 61 34 63 36 34 66 66 63 64 66 35 61 39 35 64 39 62 33 66 62 35 37 62 37 66 ff ff ff ff 01 00
[217] ce 59 4c fe f2 4e 00 17 a9 14 8c b1 c2 36 d3 4c 74 22 1f e4 16 3b bb a7 39 b5 2e 95 f4 84 87 00 00 00 00

I got to the point of parsing the size and header based on the btc instructions:

$n$blocksize
[1] 331

$n$total
[1] 339

$n$data
[1] 251

 

$version
[1] 1

$prevHash
 [1] 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

$merkleHash
 [1] fd f8 46 6d 1a 52 b6 8a c5 a2 f3 ef 2e 0e eb fa 40 f0 ba 78 0a f2 3c 8c 97 43 ab 75 1b 90 27 08

$time
[1] "2018-07-13 03:31:53 CDT"

$nBits
[1] 0

$nonce
[1] 0

Any hints on the tx data portion or at least how it may differ from btc? Perhaps just link to where its defined in the xaya code?

 

Share this post


Link to post
Share on other sites

Yes, the RPC interface is available also on Windows - you don't need to start with "-daemon" for that; instead, you need "-server".  If you use the Electron Wallet, then it is actually already enabled - as it is also used for the wallet itself.

The block format is mostly as in Bitcoin; but as you have discovered, there is a special new data element related to Xaya's innovative PoW system between what is the "block header" in Bitcoin and where the transactions start.  As you guessed, the byte following the block header is indeed a flag for the mining algorithm (nice work!).  The format of the "in between data" is described in our mining doc.  (But for the serialisation format, I typically look at the source code itself as the "ultimate reference".)

Share this post


Link to post
Share on other sites

Yep here is the answer already in the docs:

Quote

In addition, when the block is merge-mined, the highest-value bit (0x80) is also set to indicate this. This means that the only valid values are 0x81 (merge-mined SHA-256d) and 0x02 (stand-alone Neoscrypt).

Looks like there is lots of helpful info on that page already. Also got the RPC interface working now too. Thanks.

Share this post


Link to post
Share on other sites

Great!  Let me know if you have any more questions - or if you notice anything strange in the docs.  I don't think they had too many people reading through them yet. :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×

Important Information

Your use of this site is governed by our Terms of Use and Privacy Policy. We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.