<p>It turns out the main issue with the above code was that the the endianness in python was defaulting to my hex payload being mixed up. I simply needed to correct the ordering of the payload. The correct code can be viewed here: <a href="https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299" rel="nofollow noreferrer">https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299</a></p>

CC BY-SA 4.0
2020-05-04T13:01:15.830Z
2020-05-04T13:01:15.830Z
Dimitree
<p>I don't see any endpoint to get the whole transaction in Blockcypher API documentation, so I used blockstream.info API:</p>

<pre><code>&gt;https://blockstream.info/testnet/api/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1
{ 
   "txid":"2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1",
   "version":2,
   "locktime":0,
   "vin":[ 
      { 
         "txid":"382bda6ee2bac22afd051104af16146436895ecca382a76c2d66535a837254bc",
         "vout":1,
         "prevout":{ 
            "scriptpubkey":"00144ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 4ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_type":"v0_p2wpkh",
            "scriptpubkey_address":"tb1qfmxm9pn32rl3d37kuvjc4h9fak96cwkpjyyjmk",
            "value":2948920
         },
         "scriptsig":"",
         "scriptsig_asm":"",
         "witness":[ 
            "304402205e9b17758d1daf2a501a1fabe06bdbf15adaadfcf6ff80c4cfb34176bd988c0c02207450185bde5c20e06270b3b7e681e0042072c6af25fc10c62430df0f04ec5bf201",
            "023532fa6b866f37729719a3615ef3a776407048989736ac2b7fafa5678a519df3"
         ],
         "is_coinbase":false,
         "sequence":4294967295
      }
   ],
   "vout":[ 
      { 
         "scriptpubkey":"6a0e3132372e302e302e313a38313831",
         "scriptpubkey_asm":"OP_RETURN OP_PUSHBYTES_14 3132372e302e302e313a38313831",
         "scriptpubkey_type":"op_return",
         "value":0
      },
      { 
         "scriptpubkey":"a914171f697fe358f7d7238b6128930bb1fa7363b44d87",
         "scriptpubkey_asm":"OP_HASH160 OP_PUSHBYTES_20 171f697fe358f7d7238b6128930bb1fa7363b44d OP_EQUAL",
         "scriptpubkey_type":"p2sh",
         "scriptpubkey_address":"2MuMVAFK2gbTDNVBhLtpZfA4Xi62mWh6bU5",
         "value":948920
      },
      { 
         "scriptpubkey":"0014af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_type":"v0_p2wpkh",
         "scriptpubkey_address":"tb1q479t6caqp3q26a0y0pctrv9pmket5lnx24twcx",
         "value":1500000
      }
   ],
   "size":248,
   "weight":665,
   "fee":500000,
   "status":{ 
      "confirmed":true,
      "block_height":1612110,
      "block_hash":"000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7",
      "block_time":1576024567
   }
}
</code></pre>

<p>You can see here that the first output has the data is <code>3132372e302e302e313a38313831</code>, which decoding to ASCII characters seems like a local URL:</p>

<pre><code>hex = '3132372e302e302e313a38313831'
decoded = binascii.unhexlify(hex).decode('ascii')
print(decoded)
</code></pre>

<p>Prints: <code>127.0.0.1:8181</code></p>

<p>EDIT:</p>

<p>To get the transaction that interests you without using any external API you will need to run a full bitcoin node with the <code>txindex=1</code> option. If this transaction is tracked by your wallet, a pruned node would suffice. Then you can get the information about the transaction with this command:</p>

<pre><code>&gt;bitcoin-cli getrawtransaction 2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1 true
</code></pre>

<p>You should get something very similar to the answer from blockstream's API I pasted above</p>

CC BY-SA 4.0
2019-12-16T00:14:02.050Z
sorry if it wasn't clear, but I needed to do this in a decentralised way without relying on stored data from an API server (for example if they just respond with the incorrect data). I'm trying to read this data straight off the blockchain and can't find any way to do this
Dimitree
CC BY-SA 4.0
2019-12-16T00:25:11.613Z
Check the edit in the final part of my answer, it explains how to do that without an external API
Pedro
CC BY-SA 4.0
2019-12-15T19:34:03.383Z
2019-12-16T00:24:21.693Z
2019-12-16T00:24:21.693Z
Pedro
<p>The bitcoin p2p protocol does not directly permit transaction lookups this way. Maybe some bloom filtering would minimize the footprint, but it's still relatively complex to implement.</p>

<p>Electrum servers are maybe the next best thing - they do support the lookups you want, but they're less decentralized, and if your application is doing naughty things they could collude and manipulate only your results.</p>

<blockquote>
  <p>How do I get my block or headers response to only contain 1 inv item instead of 500/5000</p>
</blockquote>

<p>I don't think you can limit the headers results (you have to know all the headers to do so) But once you have the header chain, you can send a block locator with start:HASH(n) end:HASH(n+1) and you can download a single block.</p>

<p>Re bloom filtering, if you find nodes with the BLOOM_SERVICE bit you can tell them your filter, and the block data only contains your txs and the false positives allowed by your filter. </p>

CC BY-SA 4.0
2020-01-16T18:29:38.217Z
2020-01-16T18:36:34.920Z
2020-01-16T18:36:34.920Z
karimkorun
<p><strong>Background Information</strong></p>

<p>I am building an application that requires the ability to query any bitcoin full node (from a python script using raw tcp sockets) in order to read the OP_Return value listed in the following transaction (<a href="https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/" rel="nofollow noreferrer">https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/</a>) </p>

<p><strong>What I have done to date</strong></p>

<p>I have successfully managed to establish the version + verack, however I was faced with two issues. </p>

<ol>
<li>When I send a <code>getdata</code> for the necessary transaction Idit does not return the result. I have identified this is because my initial assumptions that getdata returns the necessary data was incorrect and instead it will only return transaction data for transactions still in mempool.</li>
<li><p>This lead me to my main issue, I was hoping to replicate the initial block download on my python script, starting from the block containing the transaction I'm interested in. However when I do <code>getblocks</code>/<code>getheaders</code> no matter what I seem to put in the hash filter I get a dump of 500 (or 2000) blocks/headers. Below is a hex dump of the requests I am making (just the bitcoin protocol frame):</p>

<ul>
<li><p>getblocks
0b110907676574626c6f636b00000000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5</p></li>
<li><p>getheaders
0b110907676574686561646572730000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5</p></li>
</ul></li>
</ol>

<p><strong>I am looking for the following help:</strong></p>

<ol>
<li>How can I query for transaction data without using bitcoin RPC if I can't use the <code>getdata</code> message?</li>
<li>How do I get my block or headers response to only contain 1 inv item instead of 500/5000</li>
</ol>

<p><strong>EDIT (4/5/20)</strong>
The implied requirement which wasn't made clear here was that we needed to do so in a decentralised way using the blockchain RPC api or equivalent. </p>

CC BY-SA 4.0
2019-12-14T17:45:45.057Z
You want to download 250GB of blocks from a remote peer in order to find a single output?
Claris
CC BY-SA 4.0
2019-12-14T22:38:20.007Z
No I don't want to do that, but there doesn't seem like there are too many alternatives. I'm trying to pretend like I am a bitcoin full node trying to sync (but attempting to sync from the necessary block that I need, this way I would only be syncing 1 block worth of data) I just don't think I'm using the API properly
Dimitree
CC BY-SA 4.0
2019-12-14T22:41:55.423Z
If your best choice is to do something absurd, maybe reconsider your strategy. The p2p network isn’t designed for arbitrary data access like you’re attempting.
Claris
CC BY-SA 4.0
2019-12-15T00:11:50.440Z
I've gone through a few alternatives but the main goal here is to be able to look up the data in a decentralised censorship resistant manner. So I can't rely on looking at blockchain explorer apis. Also if I use bitcoin RPC I would need to be connecting to a specific bitcoin full node which could get taken down.
Dimitree
CC BY-SA 4.0
2019-12-16T00:50:36.277Z
If you want something censorship resistant, your only choice is running your own node, and indexing the blocks in its data yourself, and then querying that.
Pieter Wuille
CC BY-SA 4.0
2019-12-17T01:05:53.203Z
is there no way for me to mock what a full node does (but only for part of the sync process)? Like surely I can say, hey I'm a full node and I'd like to start initial block download from X block and then validate the transactions in there. Is this not handled through bitcoinP2P ? Sorry in advanced if I have made any errors here
Dimitree
CC BY-SA 4.0
2020-05-04T13:02:34.120Z
@PieterWuille I might not fully understand your comment from December, but in actual fact I managed to do what I was hoping, in theory you can 'mock' a full nodes functionality perfectly fine if you simulate the initial ver-ack messages. You can view the full source code here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py
Dimitree
<full-node><transaction-verification><block-header><p2p><gettransaction>
Get transaction data from full node
Dimitree
# Get transaction data from full node

(Asked by: username_0 on 2019-12-14)

**Background Information**

I am building an application that requires the ability to query any bitcoin full node (from a python script using raw tcp sockets) in order to read the OP\_Return value listed in the following transaction (<https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/>)

**What I have done to date**

I have successfully managed to establish the version + verack, however I was faced with two issues.

1. When I send a `getdata` for the necessary transaction Idit does not return the result. I have identified this is because my initial assumptions that getdata returns the necessary data was incorrect and instead it will only return transaction data for transactions still in mempool.
2. This lead me to my main issue, I was hoping to replicate the initial block download on my python script, starting from the block containing the transaction I'm interested in. However when I do `getblocks`/`getheaders` no matter what I seem to put in the hash filter I get a dump of 500 (or 2000) blocks/headers. Below is a hex dump of the requests I am making (just the bitcoin protocol frame):

   * getblocks
     0b110907676574626c6f636b00000000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5
   * getheaders
     0b110907676574686561646572730000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5

**I am looking for the following help:**

1. How can I query for transaction data without using bitcoin RPC if I can't use the `getdata` message?
2. How do I get my block or headers response to only contain 1 inv item instead of 500/5000

**EDIT (4/5/20)**
The implied requirement which wasn't made clear here was that we needed to do so in a decentralised way using the blockchain RPC api or equivalent.

**Tags:** full-node, transaction-verification, block-header, p2p, gettransaction

> **Comments:**
> **username_1**: You want to download 250GB of blocks from a remote peer in order to find a single output?  
> **username_0**: No I don't want to do that, but there doesn't seem like there are too many alternatives. I'm trying to pretend like I am a bitcoin full node trying to sync (but attempting to sync from the necessary block that I need, this way I would only be syncing 1 block worth of data) I just don't think I'm using the API properly  
> **username_1**: If your best choice is to do something absurd, maybe reconsider your strategy. The p2p network isn’t designed for arbitrary data access like you’re attempting.  
> **username_0**: I've gone through a few alternatives but the main goal here is to be able to look up the data in a decentralised censorship resistant manner. So I can't rely on looking at blockchain explorer apis. Also if I use bitcoin RPC I would need to be connecting to a specific bitcoin full node which could get taken down.  
> **username_2**: If you want something censorship resistant, your only choice is running your own node, and indexing the blocks in its data yourself, and then querying that.  
> **username_0**: is there no way for me to mock what a full node does (but only for part of the sync process)? Like surely I can say, hey I'm a full node and I'd like to start initial block download from X block and then validate the transactions in there. Is this not handled through bitcoinP2P ? Sorry in advanced if I have made any errors here  
> **username_0**: @username_2 I might not fully understand your comment from December, but in actual fact I managed to do what I was hoping, in theory you can 'mock' a full nodes functionality perfectly fine if you simulate the initial ver-ack messages. You can view the full source code here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py  

---

## 3 answers

### Answer by username_0 on 2020-05-04. Score: 0 | Accepted answer

It turns out the main issue with the above code was that the the endianness in python was defaulting to my hex payload being mixed up. I simply needed to correct the ordering of the payload. The correct code can be viewed here: <https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299>


---

### Answer by username_3 on 2019-12-15. Score: 0

I don't see any endpoint to get the whole transaction in Blockcypher API documentation, so I used blockstream.info API:

```
>https://blockstream.info/testnet/api/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1
{ 
   "txid":"2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1",
   "version":2,
   "locktime":0,
   "vin":[ 
      { 
         "txid":"382bda6ee2bac22afd051104af16146436895ecca382a76c2d66535a837254bc",
         "vout":1,
         "prevout":{ 
            "scriptpubkey":"00144ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 4ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
            "scriptpubkey_type":"v0_p2wpkh",
            "scriptpubkey_address":"tb1qfmxm9pn32rl3d37kuvjc4h9fak96cwkpjyyjmk",
            "value":2948920
         },
         "scriptsig":"",
         "scriptsig_asm":"",
         "witness":[ 
            "304402205e9b17758d1daf2a501a1fabe06bdbf15adaadfcf6ff80c4cfb34176bd988c0c02207450185bde5c20e06270b3b7e681e0042072c6af25fc10c62430df0f04ec5bf201",
            "023532fa6b866f37729719a3615ef3a776407048989736ac2b7fafa5678a519df3"
         ],
         "is_coinbase":false,
         "sequence":4294967295
      }
   ],
   "vout":[ 
      { 
         "scriptpubkey":"6a0e3132372e302e302e313a38313831",
         "scriptpubkey_asm":"OP_RETURN OP_PUSHBYTES_14 3132372e302e302e313a38313831",
         "scriptpubkey_type":"op_return",
         "value":0
      },
      { 
         "scriptpubkey":"a914171f697fe358f7d7238b6128930bb1fa7363b44d87",
         "scriptpubkey_asm":"OP_HASH160 OP_PUSHBYTES_20 171f697fe358f7d7238b6128930bb1fa7363b44d OP_EQUAL",
         "scriptpubkey_type":"p2sh",
         "scriptpubkey_address":"2MuMVAFK2gbTDNVBhLtpZfA4Xi62mWh6bU5",
         "value":948920
      },
      { 
         "scriptpubkey":"0014af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",
         "scriptpubkey_type":"v0_p2wpkh",
         "scriptpubkey_address":"tb1q479t6caqp3q26a0y0pctrv9pmket5lnx24twcx",
         "value":1500000
      }
   ],
   "size":248,
   "weight":665,
   "fee":500000,
   "status":{ 
      "confirmed":true,
      "block_height":1612110,
      "block_hash":"000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7",
      "block_time":1576024567
   }
}

```

You can see here that the first output has the data is `3132372e302e302e313a38313831`, which decoding to ASCII characters seems like a local URL:

```
hex = '3132372e302e302e313a38313831'
decoded = binascii.unhexlify(hex).decode('ascii')
print(decoded)

```

Prints: `127.0.0.1:8181`

EDIT:

To get the transaction that interests you without using any external API you will need to run a full bitcoin node with the `txindex=1` option. If this transaction is tracked by your wallet, a pruned node would suffice. Then you can get the information about the transaction with this command:

```
>bitcoin-cli getrawtransaction 2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1 true

```

You should get something very similar to the answer from blockstream's API I pasted above

> **Comments:**
> **username_0**: sorry if it wasn't clear, but I needed to do this in a decentralised way without relying on stored data from an API server (for example if they just respond with the incorrect data). I'm trying to read this data straight off the blockchain and can't find any way to do this  
> **username_3**: Check the edit in the final part of my answer, it explains how to do that without an external API  

---

### Answer by username_4 on 2020-01-16. Score: 0

The bitcoin p2p protocol does not directly permit transaction lookups this way. Maybe some bloom filtering would minimize the footprint, but it's still relatively complex to implement.

Electrum servers are maybe the next best thing - they do support the lookups you want, but they're less decentralized, and if your application is doing naughty things they could collude and manipulate only your results.

> How do I get my block or headers response to only contain 1 inv item instead of 500/5000

I don't think you can limit the headers results (you have to know all the headers to do so) But once you have the header chain, you can send a block locator with start:HASH(n) end:HASH(n+1) and you can download a single block.

Re bloom filtering, if you find nodes with the BLOOM\_SERVICE bit you can tell them your filter, and the block data only contains your txs and the false positives allowed by your filter.


---

