Where can I find a survey on different fee estimation algorithms?

TLDR: You can find some OLD 2019 DATA comparing their accuracy for one-hour estimations here: https://bitpost.co/evaluations
I asked myself this question about two years ago and wasn't able to find an answer. I then set out to build an app that measured how accurate fee estimations from various sources were. Below are the design choices and some OUTDATED results.
The first challenge I found was that fee estimations come under many different formats, some recommend in time horizon, others in a block horizon. I made the simplification of converting all time horizon estimations to block horizon estimations. This is questionable because some estimators might be more accurate because they take into account that certain epochs are faster than others (hash rate changes). However, this resolves ambiguous situations where a block comes around the estimation horizon: suppose you make a fee estimation for 20min from now, and the second block arrives at 20min 1 seconds, do you include that block or not?
Another important consideration is "What does it mean to estimate a fee of, say, 8 sat/B for a one-hour horizon?". Does it mean that in 80% of the cases that will be enough? Or 50% of the cases? Bitcoin core estimator has for example two modes of estimation "ECONOMICAL" and "CONSERVATIVE". So, the first time in measuring the accuracy of a fee estimator should be to assign a statistical meaning to the fee estimation. I call this, finding the implicit probability of a fee estimator: the probability for which, the fee estimator achieves its highest accuracy.
Below I paste from https://bitpost.co/evaluations/help an outline of the procedure I used:

First, the application 'observes' a type of recommendation and assigns a probabilistic meaning to each recommendation type. For example, bitcoin core's 'economical' fee recommendations could mean 40% probability of being a transaction with the recommended feerate (sat/byte) be mined while bitcoin core's 'conservative' fee recommendations could mean 70% probability.

The application gathers a recommendation and hundreds of transactions that arrive at the bitcoin mempool during this period (new pending transactions).

According to the target of the recommendation (2,3,4,etc. blocks), it waits until that target is reached.

The application, by analyzing what transactions were mined and which ones weren't, derives a range of correct recommendations. If the recommendation given is out of this range, the error of the recommendation is given by subtracting from it the correct recommendation.

In https://bitpost.co/evaluations I show some of the results that I got in 2019. One evaluation means corresponds to one observation that is considered valid for one-minute. The measured fee estimators were:

WhatTheFee: https://whatthefee.io
Bitcoin core (RPC command etimatesmartfee)
Earn: https://bitcoinfees.earn.com/api/v1/fees/recommended
BitGo: https://www.bitgo.com/api/v2/btc/tx/fee
Blockcypher: https://api.blockcypher.com/v1/btc/main
BTC congestion manager: https://fees.truelevel.io/api/v1/btc/deals
Mempool.space: https://mempool.space/api/v1/fees/recommended


Why are the fee value and modified fee value always the same?

I can not see any transaction resting in Mempool having different values for Fee and Modified Fee fields?

That's probably because you haven't modified any.
The prioritisetransaction RPC exists for miners to signal that particular transactions are paid for out of band, or are the miner's own transactions that should prioritized for block construction over what is implied by their feerate.
If you're not a miner, there is no need to ever use this RPC, and the fee and modified fee will be identical for all transactions.


There is no deadline to claim these forks. If you have access to the private keys of the addresses that owned Bitcoin during these forks, you own those coins too.


Is original bitcoin program needed with multibit?

You don't need the original Bitcoin QT progam installed to use MultiBit.


Who approves the feature and bug code reviews in Bitcoin?

How do all million plus users agree to these feature/bug requests?

Mostly by deciding whether to continue using Bitcoin-core or switch to one of the many other existing implementations of a Bitcoin wallet.
Those who are particularly competent to assess the changes are probably also, in many cases, competent to fork the code base and not introduce changes they object to. If others share their view of the changes, this might be viable. Certainly Bitcoin core has been forked many times in the past.

Do all users of Bitcoin [core] have to approve them

They mostly get to vote with their downloads.

Related questions

Where can I access the original, Satoshi written Bitcoin code?
Original default bitcoin script
Do "the official bitcoin developers" control Bitcoin?
Is the Bitcoin Core software managed by a central authority?

Other relevant sources of information

https://satoshi.nakamotoinstitute.org/code/


The best place to find bitcoin's current value is probably bitcoincharts.


Is there a site (or formula) that easily calculates returns using difficulty / hash rate?

Have a look at coinwarz.com - it list's alot of crypto currencies, difficulty and includes a calculator to measure ROI when considering mining.


forrestv stopped updating the version, but he's still adding features.

Anyway, the Linux installation instructions say to just get the latest master:

sudo apt-get install git # if git is not installed
git clone https://github.com/forrestv/p2pool.git
cd p2pool
make


What is the marginal energy cost of a bitcoin transaction?

Yes, this number is the total energy consumption of mining, divided by the number of transactions confirmed.  The total energy consumption has basically nothing to do with the actual number of transactions, at least not directly.

There's an indirect relationship: since the total number of transactions (or rather, total size in bytes) that can be confirmed per block is limited by the protocol, then if there are more transactions, you would expect transaction fees to increase.  This in turn would increase the revenue of mining, and hence you would expect more miners to enter the network to compete for this revenue, increasing total energy consumption.  But since transaction fees are a relatively small fraction of total revenue (it's dominated by block reward), this relationship will be far from linear.

It's hard to say what the actual marginal energy cost of one transaction is.  One way to look at it is that, roughly speaking, every full node on the network will check the transaction for validity.  Looking at the logs from my own server as it synchronizes with the network, in one second it's able to validate blocks containing about 5000 transactions.  At full CPU, this machine draws about 15 watts, so that's about 3 mJ per transaction.  (Of course, much of this is overhead, since the machine is powered on regardless and used for other things.)  Now https://coin.dance/nodes estimates around 10000 full nodes currently on the network, so if we extrapolate from my machine, we have 30 J per transaction, or 8.3e-6 kWH.  If you make 1000 transactions, that's 30 kJ or 8.3e-3 kWH.

If you want to exclude overhead, you could suppose that if not for verifying transactions, my server would have been idling its CPU during that time, in which state it draws about 10 watts.  So the marginal CPU usage is about 5 watts, or about 1/3 of the values above.  This logic takes you to 10 kJ per transaction.  On the other hand, my computer is pretty low-power compared to most (it uses a mobile CPU), so you may instead want to increase those values.  But this should be on the right order of magnitude.


Blockchain.info: Is there a way to know if a transaction is confirmed or not?

The rawtx endpoint has a field block_height. If this field has an integer value, then the transaction was confirmed in the block at that height. If it is null, then the transaction is still unconfirmed. You can use that to know if a transaction is confirmed via the blockchain.info api.


How to get Ledger Nano S to work in an Ubuntu VirtualBox?

You need to update the udev rules (on the guest OS) with this command:

wget -q -O - https://raw.githubusercontent.com/LedgerHQ/udev-rules/master/add_udev_rules.sh | sudo bash

It works, I verified it.

source: https://support.ledgerwallet.com/hc/en-us/articles/115005165269


claim bitcoin cash with bitcoin qt wallet

First of all, move your bitcoin to a new wallet. So it stays safe and secure.

Then download Electron Cash Wallet, create a new wallet and sweep the private key(s) of your old bitcoin-qt wallet.


Transferring bitcoins from my blockchain addresses to an exchange?

Assuming you mean "blockchain.info".

Go to your exchange, deposit/bitcoins section. It should give you a bitcoin address, copy it.
go to your wallet, "send money"
enter the copied address and the desired amount
hit "send"


No, it's all the same. Well, maybe there are some very small differences, but in general it's the same as forex. I would say that the cryptocurrency is much more unstable, and this is why I prefer trading CFDs on Investous broker. I am much better in that area, and it's much easier for me to analyze this market. However, it has the same thing at base, so, if you are already good in one area, you will be successful in the other one too.  I would suggest reading more information about that on the internet, there is a lot of information available on that topic


Can one set how many confirmations MultiBit requires for a transaction to be accepted?

From your side, an outgoing transaction are completed the moment you send it. However, it's up the receiving party to decide how many confirmations they require before they consider it received. The usual rule is 3 confirmations.

I have never used Multibit, so in regards to incoming transactions, I have no idea.


Why are my Bitcoin/Bitcoin Cash commands not working on SV?

That error means that the command bitcoind was not found on your computer's path. Make sure that you have installed the software correctly such that the bitcoind binary is on the path.


You do not need append padding, this step already implemented under the hood in the cryptographic library function sha256. As I remember, when I solve the same task and read this doc from stratum pool a few inconsistencies in the documentation that had to be solved empirically.

One of the nuances on which note that miners software do not know anything about the segwit and coinbase transaction for the miner job should be represented in the old format.


As someone who was involved in doing that migration at the time, I believe it was the right decision. LevelDB is far from perfect, but I wouldn't know what else to use.

In particular:

BDB is much slower for our usage (large atomic batch writes, small random reads).
There were reports of database corruption as well with BDB, at a time when it was used far less intensively than LevelDB is now.
BDB is very painful to upgrade. My impression is that it was designed for setups where a database upgrade only happened with professional supervision. In particular, the write log files created for durability were sometimes not readable by later versions. This is the reason why Bitcoin Core releases have for years stuck with BDB 4.8 for the wallet.
BDB has many resource limits that need configuration, where inappropriately chosen values may cause network-wide failures (read up on the March 2013 incident, specifically). The official documentation on this states Reviewing the Lock subsystem statistics is the best way to determine this value. I believe that's not acceptable for our use case.

While database corruptions are reported relatively frequently these days, my belief is that it's mostly hardware failures or driver issues. Bitcoin Core tends to stress disks, memory and CPUs far more than most software, which makes otherwise invisible issues pop up.


Currently, the most secure place to store bitcoins online appears to be the exchanges. There is some evidence that Mt. Gox is also hoping to become the default bitcoin bank as they are developing mobile payment applications for instant payment.

http://mtgoxlive.com/mobile/

They have also reimbursed the losses of another bitcoin exchange.

https://www.mtgox.com/press_release_20110811.html

It is worth pointing out that while Mt Gox was hacked, no one lost any bitcoins that were not reimbursed.


This indicates that your miner couldn't get work from the pool fast enough and so some mining power was wasted. If this only happens occasionally, I wouldn't worry about it. If it happens often enough that it's impacting your mining rate, you may wish to configure a backup pool. If it's happening because your Internet connection is slow or unreliable, there's probably nothing you can do about it.


Transaction with highest fee, but stuck, what did I do wrong?

The hourly average fee when this question was posted was between 9 and 10 Satoshi per Byte (sat/B).

The latest TX's that I just randomly checked all had 10.03 sat/B fee which is higher than the 4.5 sat/B that you sent.

Transactions are at a hourly peak right now. Wait it out and it'll probably go through in the next couple of hours as the mempool empties.

I just noticed your TX went through and has 3 confirmations from the time I started replying to now. So I guess this is a closed issue.


xPrivKey is a BIP32 extended key. This can be used to derive child addresses.

importprivkey expects a WIF Wallet Import Format.

In order to convert to from the extended to compressed format, all you need to do is decode from Base58Check and grab the last 32 bytes (33 for public keys). See BIP32 - Extended Key Serialization and then encode to WIF format:

$ printf "xprv9uHRZZhk6KAJC1avXpDAp4MDc3sQKNxDiPvvkX8Br5ngLNv1TxvUxt4cV1rGL5hj6KCesnDYUhd7oWgT11eZG7XnxHrnYeSvkzY7d2bhkJ7" | base58 -dc | xxd -p 0488ade4013442193e8000000047fdacbd0f1097043b78c63c20c34ef4ed9a111d980047ad16282c7ae623614100edb2e14f9ee77d26dd93b4ecede8d16ed408ce149b6cd80b0715a2d911a0afea

The last 64 characters (32 bytes) represent the hex version of the private key.

edb2e14f9ee77d26dd93b4ecede8d16ed408ce149b6cd80b0715a2d911a0afea

Then convert to WIF:

$ priv="edb2e14f9ee77d26dd93b4ecede8d16ed408ce149b6cd80b0715a2d911a0afea"
$ # 0x80 for mainnet, 0xef for testnet
$ pub_suffix=01 # append 01 if the corresponding public key is compressed
$ mainnet_prefix=80
$ ext_priv_mainnet=$mainnet_prefix$priv
$ wif_enc_mainnet_compressed=$(printf "$ext_priv_mainnet$pub_suffix" | xxd -r -p | base58 -c)
$ echo "WIF Mainnet (Compressed): "$wif_enc_mainnet_compressed

WIF Mainnet (Compressed): L5BmPijJjrKbiUfG4zbiFKNqkvuJ8usooJmzuD7Z8dkRoTThYnAT


Is Lightning Network a blockchain technology?

The Lightning Network is a protocol that builds on top of a blockchain and uses cryptography to allow value to be transferred between untrusted parties outside of the blockchain, giving these value transfers the ability to handle increased capacity. But the Lightning network is built on top of a blockchain (hence it's known as a "layer 2" solution) because the value transferred in the network is funded by and settled to the originating blockchain.

For more information check out this talk announcing the lightning network


First dump your keys using the instructions here
Then open a new Electrum wallet, ask it to "Import private keys", and paste your private keys there.
Then you will be able to spend from there if you have coins.


How much time does it take for a Lightning node to forward an HTLC?

Short answer: Yes network Latency is should be the dominant factor here. (I have not profiled it and I am not aware of benchmarks)

If you look at bolt 02 https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#adding-an-htlc-update_add_htlc you will see that update_add_htlc message including the onion and commitment-signed message will have to be sent / exchanged. (revoke_and_ack is for settling the htlcs) while decrypting the onion and producing signatures are computationally rather expensive operations (we should not forget encrypting and decrypting the lightning messages themselves according to bolt 08 and noise_xk) these operations even on a local area network should not exceed the time for network I/O.


Why does the standard Bitcoin client not display the balance per address?

The balance of an address is almost always useless, since money is sent from random addresses.


What's a good way to "swipe" all funds into a secure wallet while hiding common ownership?

I think you're looking for a "mixing" service (I've also heard it called "tumbling"). The idea is you mix your coins with those belonging to others, and send the output to new addresses in new configurations. This makes the "taint" (percentage of coins traceable back to a single address) pretty murky.

Take a look at Mixing Services on bitcoin.it. These require a level of trust, so there's a downside there.

I think the bigger question is, why do you want everything in a single address? I've found that using a deterministic wallet like Electrum makes it so that you can have lots of addresses floating around, and you spend from those as needed.

If it were me and I wanted to empty out a bunch of old addresses into a bunch of new addresses, I'd create a new Electrum deterministic wallet, and do one large transaction where the inputs are all your old addresses, and the outputs are 20 of your Electrum-generated addresses evenly distributed amounts (ie, each gets 1/20th of the amount), and then do a second large transaction with 20 new addresses each getting random amounts. The point here being to make it look like a you're mixing coins with other people, even though you're not.

I could be wrong, so you should probably wait for a better answer on here before doing this :)


Testnet has a built in functionality that changes the difficulty to 1 if the mining process takes 20 minutes or longer.

This question explains the dropping of difficulty to 1.

So while the difficulty is 1, the blocks are being formed rapidly until the difficulty reaches to a value which takes 10 minutes.


How are popular bitcoin wallets implemented?

Accuracy not guaranteed

Blockchain.info

Wallets are json files. The json file is encrypted using AES (crypto-js) before the browsers asks that it be stored on the server.
The wallet file is embedded in the login page as a html data attribute, the browser decrypts the file with the provided password and extracts a list of private keys and addresses.
The browser then asks blockchain info the balance and list of recent transactions for the decrypted addresses.
Upon sending the browsers asks blockchain.info for a list of unspent outputs. Then selects the outputs constructs and signs the transaction itself (bitcoin-js). When ready the transaction is serialised and sent to bloackchain for broadcasting to the bitcoin network.
Authentication is done using a unique "Shared Key" contained within the wallet file rather than the users password.
Server side we use 4 servers running MySQL cluster and java. A modified bitcoind client is used to populate the database.

See How It Works

Strongcoin

Similar to blockchain.info with client side encryption.
Private keys are stored individually on the server as a database entry rather than in a single wallet file.
Authentication is done server side using traditional Sessions.
Transaction construction and signing is all done in javascript.
Uses blockchain.info or blockexplorer.com to query balances of addresses. Uses blockchain.info to broadcast transactions.
Server side uses Ruby on Rails and Heroku

Coinbase

Server side uses Ruby on Rails and Heroku
Custom MongoDB database to query balance of addresses.
Server selects outputs, constructs and broadcasts transactions.
Authentication is done server side using traditional Sessions.
The web app directly connects to the bitcoin network using Bitcoin-Ruby to populate the database.

What are they not open source?

They are commercial enterprises with a large amount of development time invested. The code being open source would a) increase competition b) cause identical phishing sites to popup c) clones maybe be hacked or run by scammers damaging the reputation of the main site.

Parts of blockchain.info are open source. I personally hope the rest of the site will be fully open sourced one day but there is much to plan. If the code is just dumped it will not benefit anyone.


verified transaction in a block is verfied by one node?

In the world of blockchain, no one trusts anyone. Transactions in each and every block are verified by each and every node.

Just the node that solves the cryptographic puzzle (In PoW), gets a chance to append a new and valid block to the blockchain. Which in turn will be verified by every node.


Bitcoin-ruby returns "nil" value for bitcoin-cli calls

The above code is only using RPC to Bitcoin Core and bitcoin-ruby is not used.

The problem is the body parameter name when sending the request.
Change param key to params as follow.

request.body = {method: method, params: param, id: 'jsonrpc'}.to_json


Why do aggregate Schnorr signatures reverse the hash and the random number

Schnorr signatures were patented by their creator, Claus Schnorr, and required permission from him in order to be used. The Digital Signature Algorithm (DSA) is similar to the Schnor signature scheme but is made available publicly for free by the US government.

DSA is largely an alternative to Schnorr and needed to be different enough from it so that it would not violate Schnorr's patent. Part of this is different signing and verification formulas.

So what you are seeing is part of the difference between DSA and Schnorr. Bitcoin currently uses DSA over an Elliptic Curve (ECDSA) and the proposed bip-schnorr uses Schnorr signatures over an Elliptic Curve. Thus the differences that you are seeing are because ECDSA and EC-Schnorr are two different algorithms designed to avoid each other's patents.


How does bitcoin reward who find the block?

A miner will create a special type of transaction known as a Coinbase transaction, as the first transaction in the block. This special transaction pays the block reward (including transaction fees) to the miner (or anyone they choose). But the miner can only put one, valid coinbase transaction in a block because otherwise the block as a whole would be invalid, and all the full nodes would reject it, making the miners work wasted because the block doesn't get onto the valid blockchain and they receive nothing. So miners are forced to play by the rules.

And yes, getblocktemplate includes the coinbase transaction if coinbasetxn is included in the capabilities list


Quantifying the long term costs associated with self hosting the Bitcoin node?

Is there a practical way of quantifying the cost associated with self hosting the Bitcoin node? This includes the cost of hosting, scaling and hiring a competent developer to setup and debug the node.

Depends on the business. If it's a startup you can always hire a developer for initial setup. If you experience any issues in future, pay another developer per hour for a specific work.

What is a practical way of quantifying the long term costs involved and what type of maintenance/upgrades would a developer be required to do in the long term?

Type of upgrades/maintenance can involve anything mentioned on https://bitcoin.stackexchange.com or https://github.com/bitcoin/bitcoin/issues


Can I recover my bitcoins from an Armory paper wallet even if Armory ceases to function?

Short answer, you need Armory to open an Armory paper wallet.

I use Armory and it does not fill me with confidence. I set one up and it was taking forever to sync with the blockchain. I think it actually crashed. However I had bought some BitCoin which was in there somewhere but not showing. I made a paper backup and formatted the hard drive because the PC was playing up.

8 months later I discovered my Armory paper wallet saved as a PDF and decided to see if I could get hold of my BitCoin. The codes did not seem to be any use with any online tools I could find. It would not import into an online BitCoin wallet. I did not want to have to download the 140GB blockchain.

Anyway I did install Armory and activated the restore process for the paper wallet. It restored fine but seemed to be empty and I wondered why I had bothered saving an empty wallet that had never had transactions. The process of getting Armory properly running is like firing up a steam engine compared to turning the ignition on a car. I had to also install the normal bitcoin-qt in order to get the block chain. This took two weeks. Even then Armory was not happy. Several restarts later Armory had synchronized it's database and all the other nonsense it has to do. That took a couple of days or so.

Finally I discovered the wallet had £150 worth of coin in it! Dead chuffed.

I am good with computers but I found all of this hard work and stressful. The average person will need all this done for them. I'd put it on the same difficulty level as formatting your hard drive and re-installing the OS and all the drivers. Not the hardest thing in the world but beyond most people.

I'd say any sort of wallet you use needs to be backed up in a way that it can easily be examined in a couple on minutes rather than the 2 or three weeks it took me. It may well be down to my lack of experience but I did not get the impression that an Armory backup could be used on anything but Armory.

As a little rant I'd say Crypto has to get much easier than currently if more than just technogeeks are to use it.


You can use zmqpubsequence to get updates of all transactions added/removed to/from mempool and blocks received.
Example:
Save zmqpubsequence=tcp://127.0.0.1:28332 in bitcoin.conf
Run below python code:
import binascii
import asyncio
import zmq
import zmq.asyncio
import signal
import struct
import sys

if (sys.version_info.major, sys.version_info.minor) < (3, 5):
    print("This example only works with Python 3.5 and greater")
    sys.exit(1)

port = 28332

class ZMQHandler():
    def __init__(self):
        self.loop = asyncio.get_event_loop()
        self.zmqContext = zmq.asyncio.Context()

        self.zmqSubSocket = self.zmqContext.socket(zmq.SUB)
        self.zmqSubSocket.setsockopt(zmq.RCVHWM, 0)
        self.zmqSubSocket.setsockopt_string(zmq.SUBSCRIBE, "sequence")
        self.zmqSubSocket.connect("tcp://127.0.0.1:%i" % port)

    async def handle(self) :
        topic, body, seq = await self.zmqSubSocket.recv_multipart()
        sequence = "Unknown"
        if topic == b"sequence":
            hash = binascii.hexlify(body[:32])
            label = chr(body[32])
            mempool_sequence = None if len(body) != 32+1+8 else struct.unpack("<Q", body[32+1:])[0]
            print('- SEQUENCE ('+sequence+') -')
            print(hash, label, mempool_sequence)
        # schedule ourselves to receive the next message
        asyncio.ensure_future(self.handle())

    def start(self):
        self.loop.add_signal_handler(signal.SIGINT, self.stop)
        self.loop.create_task(self.handle())
        self.loop.run_forever()

    def stop(self):
        self.loop.stop()
        self.zmqContext.destroy()

daemon = ZMQHandler()
daemon.start()

Source: https://github.com/bitcoin/bitcoin/blob/master/contrib/zmq/zmq_sub.py
python3 zmqtest.py
b'0000000000000020dd401c162d8514e7cc13a2aa123fbdaab05e117defe4fa45' C None
- SEQUENCE (Unknown) -
b'034c21d4ea43da1640baeed6a5b5da7d9581f946187840326a55a3fff652f896' A 1
- SEQUENCE (Unknown) -
b'507ec9296156c3925c6e2852284eeb07838480202d80eeb0718fc271e5c5a7e6' A 2
- SEQUENCE (Unknown) -
b'8da15cc34a0a16ef4160648a387afb98a212f4d93844c0b837ce156f3d40b70e' A 3
- SEQUENCE (Unknown) -
b'47134355caf0539bf1bb3c9496eb212318ed64f26bf8f7654026a7254fe8a3b1' A 4
- SEQUENCE (Unknown) -
b'044a56e511b1ff097c528cee053c8b4d0b97fe4237308f00d9714eca99ed7d58' A 5
- SEQUENCE (Unknown) -
b'a6edd6322b02e95a10890a811c6d0a8899d30874c8e2b02a017d785ee4ee5c80' A 6
- SEQUENCE (Unknown) -
b'4790472baba9c5860805fa3258d85d50bfa607f1ce8437ef2c4a3c6b503a8b86' A 7
- SEQUENCE (Unknown) -
b'ca82da8ee0e982995f7dd0a98580c102ae4adc4f8edf29ca64b98b1f3f1be751' A 8
- SEQUENCE (Unknown) -
b'1bcde3da8835980b8b884ba58bd36884fb5265fa6a06aabaabc8cf7517ce58ee' A 9
- SEQUENCE (Unknown) -
b'5a0d816ff22037c04d10597496c05bce9e72c6b379e33e24196902c64e2429dc' A 10
- SEQUENCE (Unknown) -
b'd0ddd43dafefb4230a3f71bd026bf7cf5e740d23163f95b809620fefeca9b0eb' A 11
- SEQUENCE (Unknown) -
b'a186fd5560d6def8eace6e9aa4a370c49cfcdbfb7883988697e9d008d1c81f73' A 12
- SEQUENCE (Unknown) -
b'40f790b2823b55ef21544f284e3381dcccd387c44512a2518edc0d64b7c78ff8' A 13
- SEQUENCE (Unknown) -
b'24fef1ea3a3058ec23015ae01788a9386fd937646dcc0ebc30d4078d4fd0ab83' A 14
- SEQUENCE (Unknown) -
b'245661a7e1a099b636541e80933720040b947efd8d63e96bae6a627ed5024d31' A 15
- SEQUENCE (Unknown) -
b'200e1379db62a299c4df69b2543ac9acc538cddd52bbc2000a40878a246ad5d2' A 16
- SEQUENCE (Unknown) -
b'37d8b02e67d30583ebfc41861c64a8bf0e10daec25366976ed002f8fa76e8300' A 17
- SEQUENCE (Unknown) -
b'677f74c4412d317921528809e983ccd57d1d387760614a99606d1d651bbdcb76' A 18
- SEQUENCE (Unknown) -
b'5c490f86c11c372d2cfd395020eb2a3726970f64829c7254f99520c8e9e9fa02' A 19
- SEQUENCE (Unknown) -
b'a176e21826a217cda3d3ef59393a9ba293ab7ca238f17b3a945a0928e172221d' A 20
- SEQUENCE (Unknown) -
b'7bc33a81ee5c5cd5083ef11e6a9d0d50fbe081fad0bf9d2f4c980f228a32f5a7' A 21
- SEQUENCE (Unknown) -
b'18f5d3c96a66a75d20f8f19c4a1d1eff28488610d8ac1241865e5d7b7a94bfbc' A 22
- SEQUENCE (Unknown) -
b'd14f474c19d0814c23f742f5bf7372dda29f872c072bbb7e25e97409020e4299' A 23
- SEQUENCE (Unknown) -
b'00000000000000046fbd9d36c40562094bf0049ead63e5311ac515123baf5789' C None

How to save the mempool information immediately before the node receives a block?

In the above results, Block 00000000000000046fbd9d36c40562094bf0049ead63e5311ac515123baf5789 was received and 23 transactions were added to mempool before it.
If a transaction is removed from mempool, python script will print:
b'TX_ID' R SEQ_NUMBER

Hope this helps although its not necessary that transactions between 2 blocks printed by the zmq python script will be same as getrawmempool results at that moment. The script will only print additiion and removal of transactions in sequence.


Are Bitcoin's blocks signed by the miner?

A sort of. Not a cryptographic signature, but each miner is incentivized to use one of its own BTC addresses as recipient of coinbase (reward) transaction. This links the block to the miner, with the same strength each UTXO is linked to its owner (I want to stress that there isn't native identity concept in bitcoin's blockchain, so we can say that the attribution allowed by core methods is already implemented)
Can you elaborate on signature (maybe stronger one) advantages, from your POV? Thanks


How can a Bitcoin Oracle ensure that the money can't be respent?

(elaborating on the comment I gave to the question)

You do this by wiring the funds to an address controlled by both you and an oracle (or a set of oracles). You do this by utilizing a bitcoin multisig address.

That way there's no way for you to move the funds away without oracle's permission, and there's no way for oracle to run away with funds.

As for preventing oracles from holding the transaction hostage, you can use distributed oracles - i.e. a set of independent oracles. So, to withdraw the funds you'd need a signature of both yourself, and say 8 out of 15 oracles. If you choose oracles wisely, the chance for them to collude and hold your funds for ransom should be minimal.

You can consult Orisi Whitepaper ( https://github.com/orisi/wiki/wiki/Orisi-White-Paper/ ) or website (oracles.li) or github repo (https://github.com/orisi/orisi) for more info.


The wallet doesn't have to be on the same computer. For example, you can mine in a pool and the address to receive bitcoins could be from any wallet located anywhere.


Closed a wallet in Multibit mac and cant find it from application support

Are you sure you looked in the right folder? The path should be: /User/your username/Application Support/Multibit/xxxx.wallet.

I didn't find my wallet because I made the misstake to looked directly under root: /Library/Application Support/  and got rather stressed but I found the path eventually.

This cataloug is hidden and you have to unhide it with the commando in a terminal window: chflags nohidden /Users/your username/Library/Application Support/


Calculating fee rate of tx returned by fundrawtransaction

You're seeing two things here: the exact match coin selection algorithm, and the fact that your vsize is computed before adding signatures.
Exact match
The coin selection algorithm in Bitcoin Core tries to find an "exact match" of UTXOs to spend in order to pay, without adding a change output back to you. If the input amount is a bit too large, it is turned into fee. Surprisingly, under certain circumstances this behavior is actually preferable over turning that excess into change.
The observation is that merely creating a change output costs money right now (in terms of fee needed to pay for it), and more money later (in terms of the marginal fee needed to pay for the additional space used by the transaction input that will spend it). That fee is wasteful: if it's possible to not create the change at all, this becomes a cost you never have to pay.
So the conclusion is: if it is possible to construct a transaction without change that overpays less than what it would cost to create and spend change, it is preferable. This is exactly what Bitcoin Core does, using a short-term fee estimation for the creation of the change, and a long-term fee estimation for spending it.
The concept and algorithm for doing this efficiently were first described by Mark Erhard in his master's thesis.
vsize before signatures
You're looking at the vsize of the unsigned raw transaction coming out of fundrawtransaction. For fee decision purposes, fundrawtransaction tries to predict the size of the transaction after it is signed.


How profitable is mining with a ~1GH/s miner?

Minergate is not by any means an effecient way to mine alt-coins. If you want to mine Monero XMR, you need to download either the source; and compile a GPU Miner, and configure it to mine off one of the many pools; and do the same for a CPU Miner; that plays nice with your GPU miner.

Claymore's GPU and CPU miner are widely adopted and used, I am not sure if they work on Macs however.

You can visit the bitcointalk.org thread for Monero; and get all the information you need on the OP Thread Post including links to miners; pools, exchanges, etc.

But in closing; XMR is trading very high right now. I mined XMR two years ago and accumulated a lot of coins... this was when BTC was at 400-600 and XMR was 1/15th the current price. Across my systems; I was mining 10-20 XMR a day. But that was 2 years ago and difficulty is drastically higher now.

HERE IS A LINK TO CPU / GPU BENCHMARKS FOR MINING CRYPTONIGHT ALGORITHM COINS

http://monerobenchmarks.info/

HERE IS A LINK TO THE THREAD ON BITCOINTALK WHERE YOU CAN GET STARTED

https://bitcointalk.org/index.php?topic=583449.0

In closing, you may want to look at AEON, which uses a forked version of Cryptonight, called Cryptonight-Light. It hashes at much higher rates. The coin seems to be in a bit of an unstable state development wise; issues with the new wallet release; etc, but in my experience mining the most profitable coin - at the moment - is not always the best strategy.

The key is finding the lower difficulty coins that you can mine now at a loss; that will 10x or 20x in value 12 months ahead. Finding these coins is the hard part; but that's where the money is.


How do I trace a bitcoin transaction to find its location in the world

The Bitcoin blockchain does not have any concept of location, nor could such a thing be added to it.

There is no such thing as "tracing a transaction" in a corporeal sense.

If you had a comprehensive global surveillance network, you could attempt to connect to every Bitcoin node in the network (including all the nodes located in Tor) and use timing analysis to attempt to guess the originating node of any given transaction.

If you had that surveillance capability, it would only tell about the specific outputs which were being spent and nothing about outputs which the user of that wallet might possess which have not yet been spent.

Basically, Bitcoin was designed to make what you're trying to do impossible.


getnewaddress will generate a new Bitcoin address for your existing wallet, Yes you can get as many as you like.

I would use getreceivedbyaddress which returns the total amount received by the specified address in transactions with the specified number of confirmations or listunspent to get all transactions for specific address with confirmations

AFAICT, You can't remove an address from your wallet using any RPC call, but by editing wallet.dat manually.


Basic Concept of Bitcoin (public and private) Keys

This is not complete answer, but hope this will help you to understand some concepts.

xpriv and xpub are key pairs, which are possessed by every (online) wallet when it is created. xpriv is actually the private key of the wallet (which you must not share with anyone). while xpub is the public key of the wallet (which is used to derive the receiving addresses of wallet, where it can receive funds or coins).

This is brief explanation. you can read more to understand its working and purpose from given link.
How are public and private keys related to the wallet?

In order to create wallet we use Network Parameters. So, in bitcoinj, we use MainNet (ProdNet) and TestNet. So, xpub is the public key which is used to create the public addresses of Online wallet using MainNet. while tpub is the public key, which is used to create the Receiving addresses of Testing wallet (which doesn't contain real coins).

You can check more difference between mainNet and testNet in following questions.

Practical difference between mainnet and testnet
https://ethereum.stackexchange.com/questions/6278/whats-the-difference-between-the-testnet-and-the-production-network-technical

Good Day!


The very last line of a block effectively contains:

Reward for Solving this Block: Add 12.5BTC to Account for [Mining Pool].

If the miner tried to change that line to read

Reward for Solving this Block: Add 12.5BTC to Account for [My Personal Account].

the altered line would change the block, which would change the hash, and their Proof of Work answer would no longer be valid.

So if a miner solves a block on behalf of the pool, they cannot submit that block for themselves; it will no longer be a valid solution.


What are the minimum sizes for transactions and blocks in Bitcoin?

There is no minimum size restraint on blocks and transactions. However, due to the nature of blocks and transactions, there is a practical minimum.

The smallest transaction I can think of is 61 bytes. It is a transaction that spends an OP_TRUE anyonecanspend output and creates 1 OP_TRUE anyonecanspend output. The smallest block I can think of is 146 bytes. This is the 61 byte small transaction + 4 bytes for block height in that transaction + 80 byte block header + 1 byte for transaction count.


Error: Keypool ran out, please call keypoolrefill first

Unlock your wallet (using walletpassphrase) and run keypoolrefill as the error message says. generate and setgenerate requires keys from the wallet, and if the keypool is empty, they won't work.


Transaction and Node Proximity Questions

You add the transaction T to your (the sender's) unconfirmed transactions pool. From there on, it is treated like any other transaction. Then, your client will send a "inv" message to it's peers saying it has a transaction with hash H. (H being the hash of T) If the peers don't have it, they will send a "gettx" message requesting it, and your client will send it to them. They then add it to their list of unconfirmed transactions and they will do the same thing with their peers.


Most mobile wallets are light-clients, meaning they don't download and validate the entire blockchain history. Instead, they just download the block headers (80 bytes per block), and then request any additionally needed block/transaction data that is related to the addresses that the wallet controls.

At the time of writing, the network is currently at blockheight ~623,000, so we should expect a light client to download about this much data:

623000 blocks * (80 bytes / block) = 49,840,000 bytes = ~50 Mb

The actual amount will likely be slightly higher, but we can get an idea for the order of magnitude of data from this estimate.

So, assuming that this wallet is just a light-client, it is odd that it would download 50 Gb of data. If this is the app you are mentioning, the recent user reviews make it sound like this excessive data usage is a bug (see the dev responses on that page).


Possible to create a double spend attack by replacing a Tx in the merkle tree?

You should also know that the task of finding an SHA256 collision (same hash for two different byte arrays) is a pretty complex one. Miners are spending terahashes per second to find a hash which is smaller than a target. An exact match would be very hard to find.

Also, the attacker in this task is required to be a miner with enough power to solve and broadcast a [special] block to be included in the main chain.

In short, I can say that this task requires so much hashing power - so if someone has enough power for that, he may perform a less complex 51% attack rather than this one.


Compiled Bitcoin Client as Testnet with bitcoin.conf

The location of your bitcoin.conf file depends on your operating system: https://en.bitcoin.it/wiki/Running_Bitcoin#Bitcoin.conf_Configuration_File

Note that you can also pass any configuration value as command line argument.


What Ripple API libraries are available?

Ripple labs have created a number of GitHub repositories relating to Ripple, including ripple-lib for JavaScript and a work in progress ripple-lib-java for Java.
Rubble Labs have released their ripple library for Golang
CodeShark has released a C++ library for communicating with Ripple through a websocket


You will need to buy bitcoin, then send that to your acquaintance. Other option is to convince your acquaintance to accept another payment method.

Most bitcoin sellers will not accept credit card because it is very easy to lose money from chargebacks.


How can I see or validate mining in regtest mode?

You can use the getblock <block hash> command to see the details of a block.


This depends on the wallet (keypath) you are using.

If your wallet is based on BIP44 (most wallets are), sharing the extended master public key will not allow the other party deriving pubkeys to monitor incoming funds (because of hardened key derivation). You need to share the specific account extended public key. Your fist account is at keypath m / 44' / 0' / 0', second at m / 44' / 0' / 1', etc.

Instead of sharing the pure pubkey together with the chaincode, we use "extended public keys". You can identify a such base58check encoded string by the first 4 chars "xpub".

As far as I know there are only a few wallets in the wilde who can do such exports/imports.


Maintaining Tor hidden service onion address over new hardware

Depending on the version of Bitcoin Core and Tor you are using, there is the possibility that your installation was actually making use of Tor control and not actually using the configuration from your torrc. Have a look in the /.bitcoin/ folder for onion_private_key and try exiting Bitcoin Core, putting that file back in place from your old HDD and, restart Bitcoin Core.

I do not know if you will also need to do the same for the Tor auth cookie /run/tor/control.authcookie?

There is an updated setup guide for running Bitcoin Core and Tor on Bitcoin.SE right here.


What's the difference between BTC and BCH transaction capacity?

tldr - BTC = 3.33-6.66tx/second, BCH = 27 tx/second

For non-segwit-enabled Bitcoin nodes, transaction capacity is determined by Bitcoin's 1MB block size limit. A 1MB block will contain about 2000 transactions on average (see here). Because 1 block is mined every 10 minutes, this works out to roughly 3.33 tx/second.

BCH transaction capacity is determined by its 8MB block size limit. Because a given non-segwit BTC transaction should be roughly equal in size to its equivalent BCH transaction, this puts the transaction capacity of BCH at 8x that of BTC without segwit, or roughly 27 tx/second.

For segwit-enabled BTC nodes, transaction capacity is limited by the 4MB block weight limit (Note that block weight ≠ block size, see here for more info). It is estimated that roughly 2x the amount segwit transactions can be fit into a BTC block as can their non-segwit equivalents (see here and here). This would make the transaction capacity of a fully segwit-enabled BTC network roughly 6.66 transactions/second.

A few things worth noting:

1) Visa typically processes ~2,000 transactions/second and can process up to 24,000 transactions/second (see here).

2) As of Feb 2018, roughly 14% of BTC transactions are segwit transactions (see here)

3) On average, a serialized BTC block full of segwit transactions should take up ~2MB on disk.

4) Increasing the block size limit increases the amount of disk space and RAM required by miners to operate full nodes.

5) Generally speaking, the BTC community plans to increase transaction capacity via an overlay network (e.g. the lightning network) whereas the BCH community plans to increase transaction capacity by continuing to increase the block size.


The bitcoin mining algorithm from a programmer's viewpoint

The Mining Algorithm is as follows:

Step 0 - Retrieve the hash of the previous block from the network.
Step 1 - Gather a list of potential transactions known as a "block". This list of transactions comes from the peer-to-peer bitcoin network.
Step 2 - Calculate a hash for a block of potential transactions along with a random number.
Step 3 - If the hash is more than the currently set difficulty level, then you have mined that block.
If not, start over from Step 1. Any additions to the list of transactions from step 1 along with change in the random number from Step 2 mean that there's a chance that the criterion will be met in the next go around.

From a programmer's view, the pseudo code might look something like this:

P := The hash of the previously mined block
B := A block of transactions
H := A hash function
D := Difficulty Level

0 Retreive P
1 Construct/Modify B
2 IF H(P, B, Some Random Number) > D END
3 GOTO 1

I should warn you that there are a few inaccuracies in that description, but for the most part, that should be good enough. And here are a few more useful clarifications:

What's a hash?

A hash is a function that converts data into a number within a certain range. The hash has the property that knowing it's output is essentially unpredictable (within the given range). The specific hash function used for bitcoin mining is SHA256 applied twice.

How does the difficulty level work?

This unpredictable nature of the hash function means that putting in random data (the transaction + the random number) will essentially produce a random number within a certain range. Further restricting the range of the desired output affects how likely one is to find it in a single round. This creates a way to probabilistically determine how often a solution will be found based on the number of times the algorithm can be run on the network. Specifically, when you hear the term "gigahashes" or "terahashes", this refers to the number of times Step 3 can be run. As the number of hashes per second across the entire network grows, the network automatically raises the difficulty such that a solution will be found within about 10 minutes.

What happens when a block is mined?

When a block is mined, the miner sends the block to all other miners on the network as evidence that it has found it. This block contains a list of transactions, the found hash, the specific random number, and a reference to the previous hash. As each miner receives the newly mined block, it removes all transactions that it is currently mining that exist within the block (because they've already been confirmed in the block chain) and broadcasts the block to other miners that do the same thing. The propagation happens pretty quickly.

Note: the original miner of the block gets a "miners' fee", which is a reward consisting of any unspent coins from transactions in addition to a "coinbase" reward. The coinbase reward started out at 50 bitcoins and halves after every 210,000 blocks (about once every 4 years). The coinbase reward will eventually get so small that it will be miniscule compared to miners's fees.


Downloaded bitcoin core and it says synchronising with network, how do I get it to download the blockchain?

You have not lost the money, "synchronizing with the network" means that your Bitcoin Core is downloading the blockchain!

This may take a few hours or days depending on the computational power of your computer. If you don't need them immediately, you can just wait until the synchronization is finished which is what I would suggest that you should do.

If you are anxious to check whether the bitcoins were transferred, you can look up your address (that it was sent to) on one of the block explorers.


You can't. The blockchain has just forked, but the clients are not ready. They still haven't added replay protection, so the "specs" is not ready  too. When they release the client, you'll be able to import your paper wallet private key and claim your Bitcoin Gold.

Now, just don't fall for "claim your bitcoin gold" scams.


Variables in Coinbase Transaction & Decoding I/O Scripts

What exactly is the "Previous Index" and why is the value of it FFFFFFFF and not 00000000? Since the previous transaction doesn't exist, shouldn't it be 0 just like the previous hash?

Coinbase transactions must have index FFFFFFFF, which is -1.

  From my understanding, the input script contains arbitrary data chosen by the miner when constructing the coinbase transaction and can typically be decoded into ASCII.

  For example, this example input script (genesis block) decodes into ASCII as:

ÿÿEThe Times 03/Jan/2009 Chancellor on brink of second bailout for banks

  Is this correct? And is this also used when the nonce overflows as an 'extraNonce'?

Yes, it is exactly ÿÿEThe Times 03/Jan/2009 Chancellor on brink of second bailout for banks. Any part of coinbase can be used as the extranonce.

  What is the input sequence? Why is it always 0xFFFFFFFF when the locktime is 0x00000000?

See this for details

  What data is stored in the output script, and how can it be broken down to be decoded?

It is a pay-to-pubkey output.

41 Push 65 bytes

04678AFDB0FE5548271967F1A67130B7105CD6A828E03909A67962E0EA1F61DEB649F6BC3F4CEF38C4F35504E51EC112DE5C384DF7BA0B8D578A4C702B6BF11D5F Satoshi's public key

AC OP_CHECKSIG


How do i list my ICO token created on Stellar to an exchange?

Most exchanges will only list coins which have a proven record of exchange volume and price. If you create your own token it is unlikely it will be used by anyone else, or purchased on an exchange, so it is thus unlikely any exchange will want to list it I'm afraid.


Is there any available way to discover/find the public key of a specific address?

The only way is to see if any outputs for that address have been spent before, and then looking at the spending transaction(s). Those spending transactions will contain the scripts and public keys for that address.
Otherwise, no, it is not possible. Addresses are constructed by hashing public keys or scripts, and one of the key properties of hash functions is that they are not reversible. So you cannot take the hash encoded by an address and compute what was hashed (i.e. the public key or script).


Send BTC with Electrum: which address is used?

Great question, I've always wondered this too. But im pretty sure that Electrum, by default, will send your oldest unspent outputs first. (Sorry if example is a little repetitive)

E.g. If you recieved 0.4 on address A 3 days ago, 0.2 on address B 2 days ago and 0.1 in address A again 1 day ago, it would send 0.4 from address A, then 0.2 from address B, then 0.1 from address A again.

Im pretty sure you can change this in Dev mode. I'll look into it.

As I said, in order of oldest unspent outputs (balances)
Yes, they can send from multiple addresses. But it will try to send from one address if possible.
Try the coin selection setting in settings.

EDIT - Ah, yes you can get it to spend from certain addresses. Go into your Electrum settings, and set your "Coin Selection" mode to privacy.


Can my internet provider see I'm using bitcoin core (Russia)

Yes, they can.  It is quite easy to detect an ordinary Bitcoin client; it sends unencrypted messages of a standard and unique format to well-known TCP ports.  Moreover, if they monitor all your traffic, it would not be too hard for them to link you with the transactions you send; Bitcoin tries to "cloak" your transactions by sending them out along with other people's relayed transactions.  But if they can see all your incoming traffic, then when you send out a transaction that you never received from someone else, they can conclude it is one that you yourself created.

Identifying the recipient of the transaction might be harder, unless it happens to be someone they are already monitoring;  then they could see when that person spends the coins you sent.

An encrypted VPN, or proxy service like Tor, can help with this, to some extent.  However, then you have to trust the VPN provider not to cooperate with the authorities, or to monitor your traffic themselves and use it for their own evil purposes.  In the case of Tor, there might be similar problems if you connect to a Tor node which is compromised; someone who controls all your Internet traffic might be able to arrange for this to happen.


It wouldn't be a fiat currency.

  fiat money: money without intrinsic value that is used as money because of government decree

(N. Gregory Mankiw (2014). Principles of Economics. p. 220. ISBN 978-1-285-16592-9)

No government decreed its people to use Bitcoin. Quite the contrary: One of Bitcoin's goals is to take control away from governments, whereas fiat currencies are a means of governments to achieve and maintain control over their peoples' funds.

You appear to mean to ask whether it's possible to distribute banknotes the value of which is backed by Bitcoin which is what I assume to be the question I'm responding to in my answer.

In principle, it's possible.

The banknotes have to be created by an entity which is:

able to create banknotes which are hard to counterfeit and of which it's easy to spot a forged one.
trusted to keeps its promises.
predicted to last a long time.

The 1st point requires advanced knowledge about creating verifiable documents and the ability to keep secrets.

The second point adds the willingness to keep those secrets as well as the willingness to actually pay out money in Bitcoin if asked to and provided banknotes sufficient in value. This means that the entity has to have something to lose.

The 3rd point should be trivial. If the entity disappears after giving out the banknotes, people can't exchange them for Bitcoin.

As far as I can see, these requirements can only be met by states or large companies.

However, even though it's possible, fully backed banknotes are very unlikely to ever appear again. No matter whether they are backed by gold, Bitcoin, or anything else scarce. We live in a world where barely anything is backed. The credit a bank gives you not only isn't backed by some scarce good but also not even backed by banknotes lying in the bank's vault. Yet most people seem to be fine with this.

If people's attitudes shift to more critical thinking about money, of course, partially or fully backed banknotes become more plausible. Such banknotes can be backed by any scarce good. This scarce good very well can be Bitcoin which has great advantages over gold:

It's easy to transfer.
Reserves can easily be proven to exist without endangering them. And I mean actually to be proven to exist. Not just "Here is a video someone made 10 years ago, look at all those shiny gold bars we totally didn't put in there using CGI." or anything alike.
Gold – which is a resource needed for to manufacture certain things (with far lower value than it's being currently traded for, though) – doesn't unnecessarily lie around in vaults.
Its volume can't be inflated. There is a huge amount of gold in Earth's crust. If an easy way to mine it emerges, its available volume is greatly inflated, decreasing its value dramatically.


sent bitcoin to bitcoin cash address on coinbase

It sounds like you sent your bitcoin to an address that you do not own, and is not associated with your Coinbase account.  If this is the case, it's important to understand what has happened.  Forgive me if this is review for you, but others may find it useful.

Bitcoin and Bitcoin Cash addresses are the same in structure.  These addresses are a hashed and checksummed representation of the public key of a asymmetric key pair.  In order to spend them, you need the private key.  If you have the private key, you control any money paid to the address on either blockchain.

It appears as though you copied a Bitcoin Cash address generated by Coinbase, and sent Bitcoin to it thinking it was a Bitcoin address tied to your Coinbase account.  I'm not sure how this happened, but the result is that your Coinbase Bitcoin account knows nothing about the transaction.  You transferred your Bitcoin to an address that Coinbase may not even realize exists on the Bitcoin blockchain.

Here's the problem: You don't have the private key, only Coinbase does.  Coinbase may only recognize this private key as a key to a Bitcoin Cash address, but it is also a key to a Bitcoin address.  You will need to get them to use this private key to generate a Bitcoin address that is tied to your account.  You cannot do this without their cooperation.  You don't own the private key, and therefore, you technically don't own the Bitcoin.

As a general rule, do not consider any Bitcoin held by a third party to be your Bitcoin.  It's not.  Bitcoin is owned by whoever controls the private key.  In the case of a Coinbase wallet, Coinbase owns the Bitcoin on your behalf.  They are nice enough to give you a web interface to let you use their Bitcoin that you paid for.


How to identify the sender from crypto payments

As RedGrittyBrick suggested in a comment: Give each user a different deposit address.
Think of your process as the users purchasing credits in exchange for a Bitcoin payment. You respond to each purchase request with an offer. The offer includes an invoice address. Each new offer should have a new address, so you can identify which user took what offer. Since the address is unique, you'll know which user to credit how much when you receive the corresponding payment to an address.


Transactions don't work like that, so this issue does not arise.

Inputs of a transaction don't specify addresses, they specify previous transactions; more precisely, a particular output from a previous transaction.  The data included in an input is a transaction ID and an output index.

So in the example you cite, the transaction would have to have three inputs: one for each of the two Address1 transctions, and a third for the Address2 transaction.


Can you use an external PCI expansion box for mining?

The amount of bandwidth used for mining is absolutely trivial, so bandwidth won't hurt you much here. Of course you would have to use something like this box because this one doesn't appear to be nearly deep enough for a full-length video card. Using the backplane from this box in a custom enclosure might be worthwhile.

The secondary problem you'll run into is power. This unit appears to have just enough power supply to power the communications between ports. An additional PSU will be needed for the GPUs themselves which, conveniently, typically draw most of their power from four or six pin connectors.

So in short, using this backplane in a separate enclosure with an external power supply (and a lot more airflow) seems pretty viable to me. Of course I haven't personally done this so I make no guarantees, but if someone has actually attempted this I'd love to hear about it.


Will the amount of electricity used for mining be huge if Bitcoin is widely adopted?

This is a very difficult prediction to make, but there is one factor often left out of mining cost estimations: cost of hardware. When amortized over their expected lifespans, the costs of various pieces of equipment can become a meaningfully large factor in Bitcoin mining costs. In the days to come, it seems likely that FPGA or ASIC miners will become more common since they offer tremendous power efficiency gains over GPUs. The "downside" to FPGA and ASIC mining is that it is also extremely expensive to obtain the initial equipment, which means that the cost of equipment amortized over its expected lifespan is higher than with GPU mining.

We're now seeing the first commercially available FPGA mining rigs become available, costing about $440 for a 100 MH/s rig consuming 6.8 watts of electricity. Here at my home in Las Vegas, NV it would cost about 55 cents per month to run one of these FPGA boards, and even if the device survived 10 years of operation before its death, amortized hardware cost would be $3.67 per month, absolutely dwarfing energy costs. The ATI Radeon HD 5830 based rigs I currently run cost about $6.58 per 100 MH/s in electricity alone, but a 10-year amortization only adds $0.96 per month per 100 MH/s to that cost. This means the buy-in for GPU mining is low but so is the profit and sustainability.

This puts my cost per coin for FPGAs at $2.45 and for GPUs at $7.54, though there is certainly room for improvement in those numbers and your mileage WILL vary. The important bit, however, is that FPGA mining is far more profitable regardless of hardware cost and is likely to be the next big thing, which does a lot for Bitcoin's overall energy footprint since FPGA mining consumes about 8% of the electricity GPU mining currently consumes. If FPGA mining took over right now, the Bitcoin network could grow to 12 times its current hash rate without seeing a net increase in energy consumption.

In short, Bitcoin is in its infancy still and so is mining. Just as most technologies begin at low efficiency and high cost and grow towards high efficiency and low costs, so too is Bitcoin mining. You are correct in your assumption that electricity use will likely grow with Bitcoin adoption, but the missing factor in that equation is the increasing energy efficiency of mining hardware.


Bitcoin protocol allows a chain to be undone by a longer chain

Bitcoin is a distributed system. At any given time, there are n different representations of the network, where n is the number of nodes participating in the network. These nodes have a consensus protocol (defined by the Bitcoin specification) which allows each of the nodes to maintain a state consistent with all of the other nodes on the network. The consensus mechanism in Bitcoin is related to the chain of blocks (hence the namesake, "blockchain"), essentially, the longest chain wins.

Blocks can be created by any miner participating in the network, anywhere in the world. When a block is created it needs to be propagated across the entire network, to all n nodes in the world. This can take time, and there is a chance that a new block is created before this process completes. That new block could then get propagated to nodes which haven't heard of the prior block (think network locality). When this happens your network has a fork.

The way the consensus protocol deal with this fork is by seeing which version of the chain the next block is built on top of--that chain will then be 1 block longer than the other and the nodes that see this will consider this the main chain, and the true representation of the Bitcoin ledger.

Interesting cases can come in where there is a major segmentation of the network (say an intercontinental cable is severed and India drops off the internet). Different segments of the network will have a different understanding of the state of the Bitcoin ledger, and thus, could begin building a diverging set of chains. When networks are rejoined, nodes again regain consensus by comparing chain length and choosing the longes; the shorter chain's blocks become invalid.

As time goes on, the ability of a network segmentation unseating a chain is reduced--the likelihood your transactions are on an invalid chain (i.e. the likelihood there is a longer chain being mined by a different portion of a segmented network) goes to zero.


Where can I download a snapshot of the Lightning network topology?

As far as I know currently noone explicitly provides this dataset / snapshot. However romperts recksplorer works with providing this dataset from lnd to the browser and have javascript visualize it. so you could do: wget https://rompert.com/networkgraphv2 but that is a little bit dirty.

The "easiest" way to achieve what you are looking for by doing it properly is probably to extract gossipd from lightningd. As you know c Lightning has this darmonized architecture so there gossipd should be able to run standalone, peer with nodes and download sync the network graph.

I might be interested in helping to run gossipd as a standalone tool. We could also just implement the gossip protocol ourselves as that part of lightning is pretty standalone.

Hope these two suggestions help


Does anyone get custodial control of your funds when using Lightning Network?

From my understanding on how the Lightning Network works (derived from the whitepaper and video), all parties commit an amount of funds in order to open a payment 'channel'.  The architecture itself forces the funds to be either transmitted correctly (potentially multiple transfers between the involved parties), or in case of a non-collaborating channel node, for the transactions to be pushed to the Bitcoin blockchain and therefore for the channel to close without any of the involved parties losing any of the funds (but potentially not completing the intended transactions).  The custodian is the channel itself and therefore all the participating channel nodes.  Taken directly from the explanation:

  "This entry [multi-party blockchain entry] can be closed out at any time by either party without any
  trust or custodianship by broadcasting the most recent version to the
  blockchain."


#0 what is the format? As you partly guessed, that is the "PEM" armoring of the DER encoding of the SubjectPublicKeyInfo ASN.1 type from X.509, republished for Internet use as RFC5280 section 4.1 particularly 4.1.2.7. To support multiple algorithms including new ones in the future, this structure is a SEQUENCE of an AlgorithmIdentifier which identifies the algorithm (using an OBJECT IDENTIFIER for the algorithm and various types for parameters depending on the algorithm), followed by a BIT STRING that contains the key value encoded in a format that varies depending on the algorithm. The AlgId and key format specific to ECDSA (and ECDH) are in RFC3279 section 2.3.5 although you can ignore the complicated parts about ECParameters because in practice everyone uses the simpler namedCurve OBJECT IDENTIFIER option, and the key format is just the SEC1/X9.62 point.

Here 301006072a8648ce3d020106052b8104000a is the AlgId and decodes as

    0:d=0  hl=2 l=  16 cons: SEQUENCE
    2:d=1  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   11:d=1  hl=2 l=   5 prim: OBJECT            :secp256k1

(As you've already figured out, "PEM" is just base64 of the DER, with line breaks at intervals, plus a ----BEGIN whatever----- line at the top and an -----END whatever----- line at the bottom.)

#1 encode without python? Aside from implementing (part of) DER yourself, which isn't that hard if you want, I see the page you link to talks a lot about OpenSSL compatibility: for programs in C or languages that can (easily) call C, which is many, OpenSSL can generate, read, write, and use for various operations ECC keys in this format (and the un-armored DER form). Java 7+ cryptography can do it, but the standard providers only in DER; for PEM you have to en/de-armor yourself or use http://www.BouncyCastle.org instead. I expect dot-NET can do it, but I don't use that and can't say definitely. I know I've seen perl modules for DER, but somehow I can never find things in CPAN even though that's its purpose. If you want a more specific answer, revise your question (or if it's sufficiently different, ask a new one).

#2 private key? That page says it uses -----BEGIN EC PRIVATE KEY----- which could be either of two 'legacy' formats in OpenSSL: unencrypted or encrypted. Since the examples don't show any password, I'll assume the former. If you look at a key file and see lines about Proc-type and DEK-info inserted after the -----BEGIN line but before the base64 data, it's legacy-encrypted and that's more complicated.

Without that, it's the private key structure defined by the SEC1 document at http://www.secg.org/ in C.4, which contains an OCTET STRING for the private key (actually an integer, but not encoded as such), parameters defined in C.2 which if you examine carefully are essentially equivalent to those in RFC 3279, and once again a BIT STRING containing the public key (a point).

OpenSSL also supports the "new" (as of about 2000!) PKCS#8 format for private keys. Much like X.509 SPKI for public key, this is a generic wrapper that is basically an AlgorithmIdentifier for the algorithm plus an OCTET STRING containing the key value depending on the algorithm, except that PKCS#8 also has an option for encrypting at the PKCS#8 level (as opposed to the "PEM" level). I don't know if the python module handles these formats or not, and if not, whether you would want to use them and thereby be incompatible with the python module. If you don't want to use PKCS#8 and do want to use OpenSSL, be careful to use the specific EC routines not the generic ones.


Insight by bitpay:

https://insight.bitpay.com/

Block Explorer:

https://blockexplorer.com/

Bitcoin Chain:

https://bitcoinchain.com/block_explorer

There are many more in development, but these are ones I could find that you didn't list.


How users collectively sign a transaction in a multisig scenario?

Not exactly.

1.
First, you need to create a receiving address that holds your "2 of 3 spendable" funds.
For that, you need three pubkeys (pubA, pubB and pubC).

2.
Form a multisig script/address with those three pubkeys.
You can use Bitcoin Cores call addmultisigaddress nrequired ["key",...].
All three participants should do that with the exact same order of the pubkeys.

addmultisigaddress 2 <pubA> <pubB> <pubC>
This will spitt out a multisig address.
The 2 stands for how many signatures are required to sign the input later.

If you want the redeem-script (expert users), call validateaddress <newmultisig-addr>
(there is a string object called script)

3.
Receive coins with your new created multisig address.

4.
Spending the coins/inputs requires now 2 signatures. If you use bitcoin-core, you need to create a (raw)transaction spending that particular input.
For that, you need to know the transaction-id and the output index of the received coin you want to spend.

Do something like

 createrawtransaction
      '[{
         "txid": "8b9123963a3bbeedac1469daf6111929286e57128812dc08e83a94a3c4e76e3e",
          "vout":0
       }]'
       '{"<newaddress>": <amount>}'

You will get a raw transaction which you then can sign (on your side, first signature or two required).

signrawtransaction <hex from createrawtransaction>

You can pass the partial signed transaction to other participants and let them so the same thing:

signrawtransaction <hex from partial signed transaction>

Details:
If you use addmultisigaddress you don't need to pass around the reedem-script and signtransaction's parameters are much simpler.


Correct - you will be awarded node_fee percentage of the shares found on your p2pool.

Relevant lines of code: https://github.com/jramos/p2pool/blob/master/p2pool/work.py#L197:L203


My full node seems to be rejecting tor connections

Turns out my node had auto-banned localhost (127.0.0.1) at some point. As a result bitcoind was rejecting all incoming connections from the tor service, including my friend's attempts. I removed the ban and he is able to connect.


what is the difference between a private pool or solo mining

There's no technical difference in generating blocks in pooling vs. solo mining. The only difference is that pool mining can create a more steady pay flow.

Why do people do pooled mining?

Bitcoin's difficulty can give unrealistic time frames for payout. The average computer will mine a single block in 1000+ years at current difficulty. But say I got into a pool with the average computer, I'd get a payed proportionally to the amount of work I contributed to a block being found. Although my "work contributed" may not end up being a block, it had a chance proportional to my own hash rate. In the end I'd get payed a few satoshis daily as opposed to getting the block reward (25 BTC at the moment) at some random point in the future likely to be no later than 1000+ years from now.

Why do people do solo mining?

The main reason is because they can. Certain miners have enough hash rate to get blocks in a reasonable time span. If I had a 1/144th the hash rate of the Bitcoin network I'd statistically have a 100% chance of finding a block of 25 BTC within 24 hours. There's no benefit for large miners to do pooled mining, may even hurt them because of fees.

Pooled vs. solo mining will theoretically yield the same amount of coins if difficulty and block reward never changed and you solo mined long enough. Although you should consider pooled mining if you aren't mining with a rack of gear.


Is there a miner with a controllable hashrate?

I suggest you run the miner at full load but decrease GPU core clock speed (GPU mem clock speed should always be reduced) and even more important GPU voltage.

Reducing GPU core clock speed should reduce power linearly. Also it allows you to decrease voltage further which reduces power quadratically. Doing so you can reduce noise dramatically.

If you use cgminer you can underclock and undervolt from within the program itself. If you use another miner you can use a program supplied by your graphic card's manufacturer (e.g. MSI Afterburner, atifonfig on linux) or a third party software such as AMDOverdriveCtrl (Linux). There is also my very own windows command line tool clocktweak (not free).


Bitcoind rpc AddLocal(ip:port) and Disover: IPv4 eth0: ip missing Amazon AWS CentOS

The fix was:

setsebool -P httpd_can_network_connect 1

Amazon AWS apparently limits httpd network connections initially.


Imagine we have a balance based blockchain, a transaction is produced which sends 1 BTC from Alice to Bob, this transaction is mined into a block and confirmed. Another block arrives which contains the same transaction again. This obviously isn't correct, however it's very difficult for us to create a system where this is not possible.

We either must:

Store a list of all transactions that have happened, refer to it in real time, and exclude any which have already been acted on (the balance deduced, and increased in the ledger).
Have a nonce for each address which increases every time it makes a transaction, and not allow a transaction with a lower nonce than we currently believe to be the case.

The latter, while being most sensible and the system used in Ethereum, is suboptimal. For every individual account as a consensus rule we must store all nonces for all eternity up to a maximum set by the consensus system. We can also not have out of order execution of transactions as they become exclusive actions (if 1 and 3 have happened, 2 can now never happen), this removes the possibility of storing pre-signed but un-broadcast transactions as is used in Bitcoin.

Instead, we store just a list of transaction outputs in the UTXO which have never been spent, and strike them from the database when they have been spent. Replay of an already existing transaction can not happen because it will spend an output that does not exist. This is ultimately the most scalable as spent transactions leave no fingerprint on the nodes storage, and allows for validity checks to happen in milliseconds.

Using a UTXO causes transactions to be atomic, and automatically not re-playable.


Different Wallets and ScriptSig does it help obfuscation?

The way transactions are associated with one another has to with the transaction's inputs.  When you send bitcoin, you are usually using multiple inputs, each with an address you control.  Anyone looking at the blockchain now knows that the same person controls all those addresses.  Now they can cross reference those addresses with addresses identified as "change" addresses from other transactions, and can be reasonably sure that the inputs to those transactions are also controlled by the same person.  This process can be repeated, which gives is a reasonable guess at which addresses are controlled by a single entity.

A wallet can only combine inputs that it controls, so using three wallets will show up as three controlling entities with blockchain analysis.  However, it will also reveal a lot of cross payments between these wallets, leading someone to assume they are somehow connected, possibly being controlled by the same person.

As for your other questions, as long as the addresses change for each transaction, so will the scriptPubKey and therefore the scrpitSig of the spending transactions.

Just remember, it's not the similarity of the scripts that allow blockchain analysis, but the association of the inputs and outputs.


Has anyone patched the code to experiment with the currency supply model?

There have been various experiments in this regard. The Alternate Cryptocurrnecies sub-forum is full of them. If I recall, Tenebrix and/or GeistGeld removed the halving block sizes, resulting in a coin without a cap on the number of coins.

Also, there is EnCoin, and alt whose price is supposed to be based on the cost of electricity.


If you're considering pasting your email as a public address to withdraw funds to, then I would recommend you search out some 'intro to bitcoin' or 'how to use bitcoin' videos on Youtube. I think watching a few of those will be quite helpful, at the very least they should provide you with enough knowledge to ask better questions. I'm not trying to be rude, but you're asking vague and wide questions about some very basic functions. Its OK to be 'bad at this', everyone starts there! With a bit of work, the system will seem less foreign to you very soon.

To take a stab at answering your Q more directly: Your wallet software can generate a million addresses, or more. Doing so doesn't really matter or accomplish anything, an address only 'becomes important' once you send bitcoin to it.

The coins you bought are probably sitting on an exchange service right now (I'm assuming you used something like Coinbase), so you probably want to withdraw them to a wallet you actually control. To do so, you'll install some wallet software, and it will generate some receiving addresses for you. Open up your exchange account, and paste one of the receiving addresses into the 'withdraw' field (it doesn't matter which address you paste in, just pick one from your wallet). Next you'll press submit/confirm, and after some time the exchange service should submit a transaction to the network to withdraw your coins.

Litecoin is a separate network, so you'll have to do the same thing again, but using litecoin software that will generate a litecoin receiving address for you.

If you try to withdraw to your email address (or anything else that isn't a valid bitcoin address), the wallet software will give you an error message, you will not be able to submit a transaction like that. The code is built to guard against errors.


How do you setup your own Bitcoin / Litecoin / Primecoin / Altcoin pool? Whats a good tutorial?

You may find my answer on how to start mining Bitcoins useful, as it touches on setting up your own pools:

How can I start mining Bitcoin?

Also this post goes into a bit more detail on using Stratum + Push Pool.

How to setup stratum with pushpool and VARDIFF? (litecoin)

Some info on eloipool configuration:

Managing Diff variable on eloipool software

If you wanted some more specific information let me know.


What could be the reason for one loop being significantly more power hungry on an SP20?

Copying asker's own answer, so that this question can be removed from the unanswered queue (assuming this answer gets upvoted).

  I have found out how to get rid of the problem: Setting the start volt 0.05 below max. voltage on all loops seems to have made it more stable and the problem vanished... I still don´t know how it came about but at least I found a workaround :)


In practice, are redeemScripts commonly reused?

Let's look at an example of this type of transaction:

The following example is the same 1-of-2 multi-signature P2WSH script, but nested in a BIP16 P2SH output.

witness:      0 <signature1> <1 <pubkey1> <pubkey2> 2 CHECKMULTISIG>
scriptSig:    <0 <32-byte-hash>>
              (0x220020{32-byte-hash})
scriptPubKey: HASH160 <20-byte-hash> EQUAL
              (0xA914{20-byte-hash}87)

So, let's say the user that created this script accepts multiple payments to this same script. In order to spend the Bitcoin received at the script, they must provide the same redeemScript, but one thing will change with each transaction, and that is the <signature1>. Since the transactions are not exactly the same each time, the data being signed will change, and even though it was signed with potentially the same key (could use key2), it will produce a different signature.

This isn't necessarily bad, but it is more common to see a business or vendor generate new script/addresses for each payment they receive so it can be easily tracked and verified.


Your posted transaction has a tx fee of 0.5 XTN:

Version:  1  tx hash 7f1aa3a20163a9d4058940dd6af29b3f7aefc037b2f3b21133a1413a990bc4e7  192 bytes
TxIn count: 1; TxOut count: 1
Lock time: 0 (valid anytime)
Input:
   0: mjKj3fFKUBMfYiozTw4jngpuDQRjarn62F from 32b88bf17552d1d8075e4086836fa587b40a16a0076877136144e445be0e9d65:1      2000.00000 mBTC  sig ok
Output:
   0: mrMEaWKfkJZcAN52VqNxKkF21uvf9HF9G7 receives   1500.00000 mBTC
Total input    2000.00000 mBTC
Total output   1500.00000 mBTC
Total fees      500.00000 mBTC

which exceeds the default max imposed by Bitcoin Core of 0.1 XTN (or BTC). Please try adding the -maxtxfee=<amt> option to bitcoind's command line, and report back if you're still having difficulty.


What are some resources for non-tea-leaf cryptocurrency trading strategies?

I've been trading on Poloniex for about 8 months now and Ill tell you what I've learned about trading crypto-currency:

There really is no method to the madness. You can try to follow trends, you can try to read the charts and watch markers and signals for whatever they are supposed to signal. But honestly, it just kinda does what it does.

I've found that trying to trade from exchange to exchange trying to take advantage of small price difference is tough because of how long it can take for different coins transactions to be confirmed. It's easy to miss a quick swing in price while waiting for funds to be confirmed.

Watching for news articles or coverage of a specific coin and then trying to trade on the news is also tough. Again because you can't be everywhere at once. The news might not inspire US investors but it might inspire Asia or European investors and they don't wake up until you go to bed. So it's easy to miss out on an afternoon rally in Asia when it's 3am where you live.

Here is what I have found to be helpful. The crypto currency market is a rollercoaster. It swings up and it swings down. And it does it very quickly and often. So here's what I do, base your buy and sell decisions on the average price over a period of time. I like to use two week intervals. Take the daily high for the two weeks and get an average price. Take the  daily low for the two weeks and get an average. Place a buy at about 5 to 10 percent above the average low and place a sell about 10 to 20 percent above the  average high. Then wait. Wait for it to reach the high you set and sell. Then wait for it to come back down (because it always does) and buy again.

If you do that over and over again over time you'll make a decent profit. You just have to be willing to wait a few weeks sometimes for it to reach the high your looking for.


How many consumer computers would it take to launch a 51% attack

A standard laptop CPU provides around 2-8 Mhash/sec. An ATI 5870 video card, which has one the best efficiency/cost ratios, mines at about 400 Mhash/sec, and at the moment, the Bitcoin network has a total hashing power of about 12,460,000 Mhash/sec, which equals 31,150 5870 video cards.

So someone would need to invest about 10 million USD just to buy video cards, without including the cost of assembly and other hardware like mother boards, power supplies etc. It is possible to design custom hardware to do Bitcoin mining using an ASIC, which might have ten times better efficiency/cost ratio.

But it will be always too expensive to execute a 51% attack to justify just by the profit from the security breach because the Bitcoin network hash rate gets adjusted to the value of a Bitcoin -- in another words, people buy more mining hardware when the price of a Bitcoin goes up.


Is Lightning Network vulnerable to sybil attacks?

Not true.

You have funds locked into a multisig wallet with party A, B and C. There's nothing to Sybil there: the funds have been transferred and only you and them have keys.

Then you want to transfer money to G. None of your parties has a direct link to G, but C has links to D, E and F. And G is advertised as having a link to D. So C relays your money to D and D relays it to G.

How do you know it's actually G and not some fake G? Because you have out-of-band communication with G, because G is the restaurant you are standing in right now or G is the webshop you are trying to pay. A secret token is exchanged directly between you and the real G, which makes sure we're talking to the correct wallets and any relay transactions can only be redeemed with the knowledge of the secret token.

The example above is simplified of course. In practice the goal is to have more hops between you and G, both because finding a shorter route might be hard/impossible, but also for privacy reasons (it uses Tor like routing and encryption to hide identities and IP addresses).


Armory Ubuntu/Debian 0.96.0 signed with a different key than listed on the website

https://bitcoinarmory.com is no longer Armory's official website. The official website is actually https://btcarmory.com. bitcoinarmory.com still hosts the new binaries, but they are only a mirror and everything else on the site is outdated.

Since Armory's development changed a few years ago, the actual release signing key is 4922589A as that is the new maintainer's signing key. This key is used on Armory 0.94.0 and above. The old signing key, 98832223, is the old maintainer's signing key.


What goes in to the message of a transaction signature?

The message whose numerically interpreted hash, z, is used to construct the ECDSA signature (r,s) is constructed as follows:

Start with your unsigned transaction.
Remove all scriptSigs (for example, if you have multiple txins, some of which may already be signed, remove those signatures). For your example this step has no effect since there's only one txin.
For the txin you're about to sign, find the corresponding UTXO's scriptPubKey, and set the current txin's scriptSig to be the value of the UTXO's scriptPubKey.
Serialize the transaction.
Append the SIGHASH type, serialized as a 32-bit little-endian integer (the default SIGHASH_ALL has a value of 1).

The resulting byte stream is double SHA-256 hashed and interpreted as a big-endian integer (and used for ECDSA's z parameter).

For your example, the scriptPubKey to insert comes from the UTXO at txid bf6c1139ea01ca054b8d00aa0a088daaeab4f3b8e111626c6be7d603a9dd8dff index 0, specifically it is OP_DUP OP_HASH160 d951eb562f1ff26b6cbe89f04eda365ea6bd95ce OP_EQUALVERIFY OP_CHECKSIG. Serialized, this is 76a914d951eb562f1ff26b6cbe89f04eda365ea6bd95ce88ac. The transaction constructed after completing step 3 is:

"txid" : "a80d616ca30a003448157b92df511ad5294e225fd77fc3f2d5dc367a4d27f375",
"version" : 1,
"locktime" : 0,
"vin" : [
    {
        "txid" : "bf6c1139ea01ca054b8d00aa0a088daaeab4f3b8e111626c6be7d603a9dd8dff",
        "vout" : 0,
        "scriptSig" : {
            "asm" : "OP_DUP OP_HASH160 d951eb562f1ff26b6cbe89f04eda365ea6bd95ce OP_EQUALVERIFY OP_CHECKSIG",
            "hex" : "76a914d951eb562f1ff26b6cbe89f04eda365ea6bd95ce88ac"
        },
        "sequence" : 4294967295
    }
],
"vout" : [
    {
        "value" : 0.49990000,
        "n" : 0,
        "scriptPubKey" : {
            "asm" : "OP_DUP OP_HASH160 7865b0b301119fc3eadc7f3406ff1339908e46d4 OP_EQUALVERIFY OP_CHECKSIG",
            "hex" : "76a9147865b0b301119fc3eadc7f3406ff1339908e46d488ac",
            "reqSigs" : 1,
            "type" : "pubkeyhash",
            "addresses" : [
                "1Bybuago2EGrB7Z6jJG2GAFDojp6Njr8fa"
            ]
        }
    }
]

It may look strange to have a scriptPubKey-style script in the scriptSig field, but that's the way it is. Once serialized, this becomes:

1: 0100000001ff8ddda903d6e76b6c6211e1b8f3b4eaaa8d080aaa008d4b05ca01ea39116cbf0000000019
2: 76a914d951eb562f1ff26b6cbe89f04eda365ea6bd95ce88ac
3: ffffffff0170c9fa02000000001976a9147865b0b301119fc3eadc7f3406ff1339908e46d488ac00000000
4: 01000000

Above, line 2 is the serialized scriptPubKey, and line 4 is the appended SIGHASH_ALL. This full message is then hashed and interpreted as an int, resulting in z=78289050778760245857840977078575435990304898491073736369300700378208907476567, which along with the private key is used to finally create the ECDSA signature.

Note that step 2 above assumes a SIGHASH_ALL signature. Other signature types remove additional parts of the transaction before signing, please see the SIGHASH types link for more details.


Link to Algorithmic behind signatures verification

Signature generation:

xRandSignPoint, yRandSignPoint = EccMultiply(Gx,Gy,RandNum)
R = xRandSignPoint % N
S = ((HashOfThingToSign + R * privKey) * (modinv(RandNum,N))) % N

Signature verification:

w = modinv(S,N)
xu1, yu1 = EccMultiply(Gx,Gy,(HashOfThingToSign * w)%N)
xu2, yu2 = EccMultiply(xPublicKey,yPublicKey,(R * w)%N)
X,Y = ECadd(xu1,yu1,xu2,yu2)
if R == X:
    print "Signature verified"

EccMultiply – elliptic curve point on scalar multiplication.

ECadd – elliptic curve 2 points addition.

Here is the original source.


How to create raw transactions with custom fees?

so I should only use the same UTXO generated by each user when depositing btcs Or is it possible to use other UTXOs as long as the quantity is the same?

This is up to you, although it would probably be safer to only spend the users deposit UTXOs to make sure nothing has gone wrong (e.g. private key leak and the coins have ready been spent). But as long as the user gets the right amount of bitcoin at the end, it doesn't really matter.

  I have read in bitcointalk that the inputs of a raw transaction must be less than or equal to the outputs of that transaction. Is it not possible to use inputs higher than the outs and "return" the surplus funds again to a wallet of my node?

I believe you have misread the information you found, or it was wrong. The inputs of a transaction must be greater than or equal to the outputs, otherwise you would be somehow generating new coins. If the inputs are strictly greater, the difference is given to the miners of the transaction as a fee. To "return" the surplus funds, you need to simply make a transaction output which sends the coins back to you, known as change.

If you generate a new transaction manually using bitcoin RPC calls, it will automatically estimate the fee itself. To use a custom fee rate, you can set the feeRate in the FundRawTransaction call. You can also set the transaction rate for all future transactions while the wallet runs using SetTxFee.


How are the nBits of a block calculated?

nBits is a compressed representation of the target value that the block's hash must be less than. Bitcoin does not actually care about difficulty. Difficulty is just for humans to think about how much work is actually being done. Bitcoin only cares about nBits and the target value it represents. A block's hash must be less than that target value. nBits is recalculated every time the difficulty changes. The calculation takes the previous target (previous nBits) and scales it by the ratio of the actual timespan it took for that difficulty period to the timespan that is targeted for a difficulty period.
What it code does is (in pseudocode):
new_target = last_block.nBits.uncompress();
new_target *= actual_timespan;
new_target /= target_timespan;
this_block.nBits = new_target.compress();

So nBits starts at the value of the genesis block, and then scales up and down depending on how quickly blocks are being mined.

The relationship between nBits and difficulty is inverted. As nBits represents a smaller and smaller target, the difficulty value goes up. The computation is essentially just the ratio between the current target value and the maximum target value, which also happens to be the target value found in the genesis block. The difficulty is the current target divided by the the maximum target.


Is it possible to view unconfirmed transactions with the bitcoin client?

The RPC interface includes the getrawmempool command which returns transactions in your memory pool. Transactions stay in your memory pool until they're confirmed, so this is effetively a list your known unconfirmed transacions.

Since getrawmempool returns only a list of transaction ids, you'll have to getrawtransaction <txid> for each transaction id in your memory pool. Remember you can get JSON-formmated output via getrawtransaction <txid> 1.


If I understand correctly, you not only do not know your private key anymore (and probably never knew), you also do not even know the BTC address where your Bitcoins are?

I'm afraid I have to say that under these circumstances it is as if you never had any Bitcoins. I lost some exactly this way as you describe and actually also had purchased them in 2011.

No chance. Best thing you can do is to forget about them and not to burden your mind with the loss.


Conflicted transactions in blockchain reorganizations

I know you're looking for some magic constant, but it doesn't exist.  Your code needs to deal with reorgs of any depth back to the genesis block.

This may seem like a annoyance of dealing with the block chain, but it's how payments have always worked.  Even if some guy pays you with a suitcase full of verified gold bars, a court order can compel you to return those bars at any later point.  (For example, it's fairly common for the court to order stolen property returned to its previous owner without any compensation to its current owner even if the current owner didn't know it was stolen when he bought it.  This notably happens occasionally with art stolen 70 years ago during WWII.)

If you really want to take a shortcut and leave dealing with long reorgs up to some future programmer or database admin, then any arbitrary constant will do.  This is why many high-value sites require six confirmations---not because six is the correct value, but because approximately one hour of confirmations is a nice arbitrary value.


My understand of this statement is that I use my 1.2 Bitcoin to buy
  1.3X1.2 Litecoin. Is this correct?

The quantity parameter refers to how many of the coin you want to buy, and the rate parameter as to the limit price you want to buy/sell those coins at.

Example, buy 100 dogecoin at 0.00000400 BTC each.

https://bittrex.com/api/v1.1/market/buylimit?apikey=api_key&market=BTC-DOGE&quantity=100&rate=0.000004&nonce=nonce

  If I want to use my Litecoin to buy Bitcoin, do I need to change the
  market from BTC-LTC to LTC-BTC??

No, the market stays the same, just change part of the URI from /buylimit? to /selllimit? or vice-versa. Selling your litecoin for bitcion is the same as buying bitcoin with litecoin.

You can make it act like a market order by checking the current bid/ask and making your rate equal to the side your trading. For example if you want to market buy you can query the best ask price and then make that the rate.

function fetchPairPrice($pair){
    $pair = strtoupper($pair);
    $uri = "https://bittrex.com/api/v1.1/public/getticker?market=BTC-".$pair;
    $ch = curl_init($uri);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $execResult = curl_exec($ch);
    $obj = json_decode($execResult, true);
    return $obj;
}

function bittrexbuy($apikey, $apisecret, $symbol, $quant, $rate){
    $nonce=time();
    $uri='https://bittrex.com/api/v1.1/market/buylimit?apikey='.$apikey.'&market=BTC-'.$symbol.'&quantity='.$quant.'&rate='.$rate.'&nonce='.$nonce;
    $sign=hash_hmac('sha512',$uri,$apisecret);
    $ch = curl_init($uri);
        curl_setopt($ch, CURLOPT_HTTPHEADER, array('apisign:'.$sign));
    $execResult = curl_exec($ch);
    $obj = json_decode($execResult, true);
    return $obj;
}

    $symbolRate = fetchPairPrice("DOGE");
    $fetchPrice = $symbolRate["result"]["Ask"];
    $amountToBuy = 100;
    bittrexbuy($apikey, $apisecret, $symbol, $amountToBuy, $fetchPrice);

    //response
    //{"success":true,"message":"","result":{"uuid":"b5f891ab-a7b4-44aa-9488-94986d1d9551"}}


How to make a "Donate" function with bitcoins?

You can just take any existing receiving address or create new one. Any address can accept unlimited transactions count.

Bitcoin clients generate new address every time you want to receive for anonymity reasons. In case with donations you don't need this.