These are ALL the notes I've taken off the Plebnet channel, or elsewheres, in very, very sparse order. Peruse them as you would like.
INDRA BOOTSTRAP NODE SINGLE BITCOIN TRANSACTION
Update: Glad you find it useful :) Yes, I would still recommend it but with a couple of modifications: 1) an onchain transaction to the LND wallet is needed to deposit at least 100,000 sats to ensure that all channels that will be opened could be anchor channels as it's so much better than not anchor channels, and 2) rather than 2M sats channels, we should open 2.05 or 2.1M sats channels, otherwise with only 2M sats channels, due to the reserve, there can never be more than 1M sats on both side of the channels, which leads to less 'good peers' stats on Web Terminal. With 2.05M sats channel, even once the reserve is taken out, there are still more than 2M sats channels lefts and tehrefore a balanced 50/50 channel will have >1M sats channels on both side and earn 2 good peers, one for inbound and one for outbound. Ingredients:
Electrum Desktop (edit: or also Sparrow Wallet, tutorial here)
12M sats in your cold wallet (as an example here, could be anything of course)
Lots of love, always needed
Do not send sats to your LND wallet on your node! (you're saving one onchain transaction). (Edit: except if you want to open anchor channels, in that case you need at least 1 UTXO with a minimum of 100k sats in your LND on-chain wallet for the anchor channel opening process to succeed).
Negotiate to be part of four 2M sats triangle on this sub (https://www.reddit.com/r/TheLightningNetwork/comments/n7s4px/lightning_network_triangle_megathread_build_the/). Make sure you let your peer know of your plan as they might be worried that you are a new node with no channels yet!
Four triangles will link you to 8 'routing' peers, try to make sure that at least a couple of them are 'good node' on Terminal Web. The goal is to have at least 3 good peers, but you'll get some in the next step.
Find 2 big 'merchant' node (e.g. exhange, merchant, wallet etc). Try to read Reddit or TG LN threads to find good nodes that will probably be quite active. Note that some of them have minimum channel size so they might not be suitable for a 2M sats channel. Take your time to read and get information to try selecting good merchants. But in the hand, it's also a question of luck! Try to get merchants that are 'good nodes' with 5 green ticks in Web Terminal.
In in text file, make a list of the pubkeys of all the nodes you will open to (i.e. 4 peers as part of the triangles and 2 merchant peers = 6 pubkeys)
Batch open transaction with balanceofsatoshi and Electrum Desktop. Below is a quick description of the process, but this step involves an onchain transaction and therefore possible loss of funds if mistakes were made. For more explanations and screenshots, I recommend this detailed tutorial on Jestopher's website: http://satbase.org/bos-open/ (there is a small paywall to pay with a LN wallet).
In Electrum Desktop, go to Tools > Preferences, under the 'Transactions' tab, actiavte on 'Advance preview'. Then in Tools, open the 'Pay to many' dialog box.
On your node, as a bos user, in the command line interface, enter this command: bos open <node pubkey 1> --amount <channel size in sats 1> <pubkey 2> --amount <channel size 2> <pubkey 3> --amount <channel size 3> <pubkey 4> --amount <channel size 4> <pubkey 5> --amount <channel size 5> <pubkey 6> --amount <channel size 6> After pressing enter, a 10 min counter will start, and you will need to do steps 3 to 5 within the 10 mins. Make sure to not use Ctrl+C once the timer has started! If you want to cancel the process and timer, just press Enter in the command line interface.
Do not enter any other command in the CLI, but just copy the output of the 'bos open' command, which will be a list of onchain addresses and amount in bitcoin separated by a comma. This is a format compatible with Electrum 'pay to many' option.
In Electrum, paste this list in the pay-to-many dialog box, save the transaction, click 'Pay', set a fixed fee as low as possible (do it during the weekend during a period of low activity, if possible <10 sats/vB). Then click 'Finalize' and 'Sign' it using your hardware wallet (or Electrum wallet if not using a hardware wallet) but DO NOT broadcast it! This will be done by balanceofsatoshi.
Once finalized and signed, copy the signed raw transaction and paste it in the command line interface and press 'Enter'. After a few seconds or minutes, bos will display a transaction ID. You can then use your node's own block explorer to check the status of your batch transaction.
Once your channels are opened and your triangle peers have also opened their channels. Ask all your peers to set up the fee rate between all your nodes to 1ppm for 15-30 min, and try to rebalance all your channels 50/50 using: bos rebalance --amount 1000000 --in <triangle 1 peer 1 node pubkey> --max-fee-rate 5 --out <triangle 1 peer 2 node pubkey> (and similarly with your 3 other triangles). You could try to rebalance a couple of triangles and ask your peers to rebalance the last 2 to share the burden. With 1ppm fees and max fee rate set at 5 ppm, it is highly likely that the rebalancing will use only 2 hops and go through your peers only, so you should end up paying only 1-2 sats to rebalance each triangle.
Only 1 onchain transaction = big saving on onchain fees by not sending sats to your LND wallet and by batch opening your channels
10 channels (this is often the minimum requirements to have some channels with some other nodes (e.g. https://lightningto.me/) and seems to be enough to be judged as having 'plenty of channels' by the various scoring system.
At least 3 good peers = If properly selected, you should be connected with at least 3 good peers that have 5 green ticks on Terminal Web. Three seems to be the minmum number required for yourself to get the 'Many good peers' green tick! (but only if you already have the first four criteria green).
20M sats capacity with 8M inbound and 12M outbound, including 4 balanced channels (1M/1M sats) if you did step 6 successfully with your triangle peers.
Activate 'anchor channels' on your node and ask your peer to do the same -> it increases the capacity of your channels and might decrease closing fees in some cases (edit: as pointed out by u/PVmining, part of the anchor channel creation requires the ring-fencing of a portion of the LND on-chain wallet for fee bumping during potential force closing of the channel... If there is no funds in the LND wallet, as suggested in the guide, it might not be possible to create anchor channels. Looking for clarification on the topic, stay tuned).
Do not register your node on 1ML, as you'll have to open a channel which means paying onchain fees, locking some liquidity and the channel will probably never be used from routing. Instead, register your node on amboss.space by simply signing a message.
Change the default fee rate of your lnd.conf file (default is 1ppm) to >10 ppm for your routing peers and possibly higher for the merchants. Setting up the fee rate is a question of trials and errors and is depending on the type of peer, flows of sats etc but it is often safe to assume that 1ppm is way too low to maintain a routing node (IMHO), although some keep it at 1ppm for ideological reasons (to keep routing fees for LN users super cheap).
[In reply to M] for now you can do the following (my suggestions) 1. Set base fee = 9999 sats (9999_000 mSat) on all channels where your local balance is < 10% channel capacity 2. Set fee rate on all channels where you purchased inbound to > 599 (or whatever you paid in ppm to buy that liquidity) so that you can recover your cost partially or fully. 3. On all channels where you have outbound >10% but < 50% keep a fee rate > 299ppm 4. On all channels where you have outboung > 50% keep a fee rate of 99ppm (or lower)
Except 1, which is #9999BaseFee, other numbers are just suggestions.
we can review your routing performance in 7 days.
[In reply to VS] [local<10%] chan.min_ratio = 0 chan.max_ratio = 0.1 strategy = static base_fee_msat = 9999000 fee_ppm = 0
[local>10%-50%] chan.min_ratio = 0.1 chan.max_ratio = 0.5 strategy = proportional base_fee_msat = 1000 min_fee_ppm = 300 max_fee_ppm = 600
[local>50%] chan.min_ratio = 0.5 chan.max_ratio = 1 strategy = static base_fee_msat = 1000 fee_ppm = 99
like that? (in the words of charge-lnd)
Here are my thought on Node P&L
Net Sats = Earned - Paid - Chain-fees - PaidToBuyLiquidity - LoopOutCustodialFees
Earned = Sats earned in a period ; use-> bos chart-fees-earned —days 7
Paid = Sats paid for rebalancing, etc. ; use-> bos chart-fees-paid —days 7
Chain-fees = amount paid for on-chain fees close/open ; use-> bos chart-chain-fees —days 7
PaidToBuyLiquidity = example if you paid LNBIG 15k Sats to open an Inbound channel to you
LoopOutCustodialFees = any Strike or Wallet Loop-out/Loop-in cost or Fees for transfer of funds to Node on-chain wallet
If the Net Sats numbers every week is improving and moving towards positive (making profits)....you are doing good. If going other direction not so good and analyze why it is negative.
Once you have the measurement system in place for your node....you can then begin to benchmark how much +ve or -ve your peers are at - if someone is willing to share.
You can also run the #s for days 30 or 60 or 90
Set the fee so that if the channel liquidity moves from the start (usually all local) to either extreme (probably all remote), it will pay for the chain fees that it took to set up this system. Chain fees include both opening the channel and closing the channel. The routing preference for LND is not based on the cheapest route, IT IS BASED ON THE LIKELIHOOD OF ROUTING SUCCESS. So, don’t worry about charging a lot; worry about not charging enough. Okay say your channel is 1 million satoshis. In order to pay for the chain fee of say ~200 sats and a channel close fee of the same amount, you’ll need to set your base fee to a number greater than zero (doesn’t really matter) and your rate fee to (200 sats open + 200 sats close)/ 1M capacity = 400 ppm. PPM is parts per million, which is parts fee per million sats routed. You can already see with this thinking that you want bigger channels, so the chain fees don’t negate your routing activity income. If you went with a 2M sat channel, you’d only need to charge half that rate to break even.
osito, [31.08.21 09:43] speaking only for my node: I do channels up to 10M and set proportional fees in categories of < 1M, 1-4M and > 4M <1M: fee rate 200 - 300ppm 1-4M: fee rate 250 - 300ppm >4M: fee rate 300 - 600ppm this is depending on channel balance (low if 100% local balance and high fees if 100% remote balance) plus individual fees for certain nodes
yeah, [31.08.21 16:35] [In reply to Jonas] (About opening 2 channels with same node, one 100% local, one 100% remote) Lnd will choose the channel with liquidity to route through, so you won't necessarily get temporarychannelfailure if one of the channels does not have liquidity.
There are a few problems with this though. If you set different fees on these channels, Lnd will pick the highest fee channel, but the client chose the lower fee ofcourse, so you'll return insufficient fee a lot.
c-lightning cannot handle multiple channels per node pair very well, causing errors. (In c-lightning it's not possible to open multiple channels to the same node) VS, [31.08.21 16:51] [In reply to yeah] If you set different fees on these channels, Lnd will pick the highest fee channel, but the client chose the lower fee ofcourse, so you'll return insufficient fee a lot. -> this will not happen (the client aka the sender only sees the highest fee on the graph)
VS, [31.08.21 17:27] [In reply to Zap] both options are available on wiki - disable is applicable only from lnd 13 onwards
VS, [31.08.21 17:30] disable could be superior - I had infact opened an issue with lnd-rpc as well that the protocol updates the disable flag at protoccol level but the suggestion was to implement it in tools outside (which charge-lnd has done) my personal preference is disable (which I have updated on my charge-lnd config)
03d1e805c, [01.09.21 10:45] [ Photo ] tbh running bos reconnect frequently seems to def make a difference, have it every 12 hours maybe should set to 6
VS, [01.09.21 10:46] [In reply to 03d1e805c] Many of us do every hour
mainnet testnet at same time https://number1.co.za/running-a-mainnet-and-testnet-on-the-same-bitcoin-node/ https://bitcoin.stackexchange.com/questions/87854/running-lnd-on-testnet-and-mainnet-at-the-same-time-on-the-same-pc#88698
DUAL FUNDED (OLD, NEW BOS DOESN'T REQUIRE DOUBLE SHELL ANYMORE) We used Balance Of Satoshis
It requires both nodes to support keysend or AMP (keysend is getting deprecated by AMP)
Run: bos open-balanced-channel
It prompts you to enter remote node public key
enter full channel size
enter fee rate, it will give you an address and amount to send to
In a separate shell, run: bos fund --fee-rate <fee> <address> <sats>
copy the signed transaction to bos prompt
Run: bos open-balanced-channel (it should see the request form node1 at this point)
It prompts you to agree with the funding rate (y/n)
In a separate shell, run: bos fund --fee-rate <fee> <address> <sats>
copy the signed transaction to bos prompt
hit enter and this should work
One thing worth noting is that using dynamic channel policy manager somewhat conflicts with the logic used by the rebalance-lnd tool - it assumes that your channel fees will remain the same for the forseeable future when it calculates the opportunity cost you're paying by moving liquidity around
HTLC - Hash Time Lock Contract is a dual lock, on one side it opens at expiry of time, on other side it opens by hash. when A forwards to B, the Time Lock is on A's side and Hash Lock is on B's side. If the payment is successful, B gets the hash, it will open the lock and prove to A that it can open the lock to claim the funds. A need to issue a new commitment transaction to settle the balance (and the HTLC) so that channel balance can be updated (funds moved to B ) however if A does not respond, B has time running against it. If A does not respond before the expiry of time lock, the funds will move back to A and B will be out of pocket. So before the expiry of timelock and A not respondign, B force clsoes the channel automatically and publish the has to claim the HTLC on-chain.
A (or A's watch tower) can dispute that if A has valid reason (and revocation key) but usually that is not the case and after expiry of to-self-delay the valid funds move to B.
Testing Tester, [05.09.21 15:50] [In reply to VS] great explanation, going to copy that into my notes .txt, I read that "B force closes the channel automatically" so it's built in lnd without having to actually do anything, and that's why people sometimes report channels force closed apparently by themselves? Also, if the timelock was to run out, would it mean that B would be losing 100% of the sum that was destined to him?
VS, [05.09.21 16:05] [In reply to Testing Tester] only if destined to B (which is why B would force close) If B indeed does not have hash to claim (payment failed and failure not propagated down the path), the time lock will open and A will claim their legitimate funds. The second scenario happen more often then forceclose. I get 1-2 stuck HTLC in a day which resolve within hours and some end up waiting for the timelock. I keep my timelock low (20 block) so I have at more risk of force close (if I am not responsive) but at the same time I get my funds back in 20 blocks.
A node which is not stable or wants to leave more lee way, can keep higher time lock
VS, [05.09.21 16:05] usual defaults are 40 only if destined to B (which is why B would force close) If B indeed does not have hash to claim (payment failed and failure not propagated down the path), the time lock will open and A will claim their legitimate funds. The second scenario happen more often then forceclose. I get 1-2 stuck HTLC in a day which resolve within hours and some end up waiting for the timelock. I keep my timelock low (20 block) so I have at more risk of force close (if I am not responsive) but at the same time I get my funds back in 20 blocks.
A node which is not stable or wants to leave more lee way, can keep higher time lock
VS, [05.09.21 16:05] usual defaults are 40
if you're worried about too much inbound, sell it to LOOP. open a channel to LOOP and if you set up your fees right you'll quickly get rid of some of your inbound and take some good fees for it.
03d1e805c, [9/7/21, 12:28 PM] [In reply to LN Eagle] loop is highly desired destination for routing, so opening a channel with them quickly moves balances to remote side after which point they close channel taking away about that channel size of inbound liquidity
03d1e805c, [9/7/21, 12:29 PM] can set channel fee as high as 2k ppm or more to them too
TomaZZ: Hey fellow Plebs, I'm planning to get a UPS back-up for my Umbrel node. I read info about UPSs on plebnet wiki and have one question. It is recommended there to NOT plug your modem/router along with your node to the UPS unit due to potential corruption if my node is writing to the disk if power outage occurs. I don't understand what plugging modem/router has to do with potential of data corruption. Is there anyone who can clarify this? I think it would rather make sense to have all equipment plugged in so that the node can stay online during short power outages? Ngu: You want to do this, so that your internet connection is not live when the battery dies, thus you are not writing to the channels.db and there is a lower risk of data corruption. If you have a usb connected from the UPS to the raspi, you can have it shutdown gracefully, in which case, you would want the modem/router on the UPS as well.
VS, [08.09.21 09:57] Channels with low flow could also be to do with liquidity in wrong direction.
Flights to Kabul may go empty but there are plenty for flights from Kabul.
Sometimes it pays to takes the cost of sending an empty plane (rebalance)
Full Throdl Hodl, [09.09.21 20:23] [In reply to VS] Maybe you could give me an idea of how you knew the direction our channel would flow before much activity occurred?
It would allow me to do the analysis myself rather than take peoples word for it
VS, [09.09.21 20:24] run rebalance in loop and watch the paths - spend 12-16 hours a day watching bos rebalance outputs on various combinations 😄
I am in favour of probing but not for free. That way those who do not want to be probed can raise up their fee (and those who don’t mind probed can bring down their fee). The probe blocks htlc on your node and many a times they get stuck - which means that portion of your channel is unusable for a long time
Probes can also be misused to make a channel temporarily disabled by filling up with probe payments which do not settle and time out.
Testing Tester, [12.09.21 08:40] [In reply to Jason] But how could you make sure channel.db was actually not corrupted, before starting lnd? yeah, [12.09.21 08:44] [In reply to Testing Tester] You could run chantools compactdb on a backup https://github.com/guggero/chantools/blob/master/doc/chantools_compactdb.md
Testing Tester, [12.09.21 08:46] [In reply to Nitesh ₿⚡️] Apart from unbalancing, if every node used max-htlc and or progressive fees and or 9999 base fee, do you think those routes would not fail anymore?
VS, [12.09.21 08:50] [In reply to Testing Tester] Provided gossip is synched
Disabled channel would not be tried so not failed. But can no act your node reputation and also can cause confusion in peers.
Max HTLC would not be tried so not failed
Fees (#9999BaseFee) would be way down below in path so less likely to be tried.
Testing Tester, [12.09.21 08:52] also another Q: will a pending HTLC influence the channel balancing until it's resolved? If I forwarded 100k and I'm still waiting for it to get back, could another forward trigger a failureof those 100k made it impossible for it to route, hence flagging me as "bad node" because of someone else's fault? And is there a way to intervene by setting max-htlc to lower when a pending htlc is present? (sorry if I don't replt right away I am going back and forth with the past history) VS, [12.09.21 08:56] [In reply to Testing Tester] Pending htlc impact future forwards until resolved. It lowers the local balance.
Any mitigation techniques would work.
Testing Tester, [12.09.21 09:30] [In reply to VS] I didn't think looping out would be a fast way to balance excessive outbound to inbound, I thought it was just to getmore onchain funds, placing it to my notes
Giovanni, [12.09.21 10:48] [In reply to Testing Tester] Let's say you want to pay D, and to do that you have to route through A, B and C.
Let's say all channels have 2M capacity, and are all perfectly balanced at 1M. You want to send 300k sats. After you have sent it, A, B and C change their max_htlc on their channels to 700k sats.
If someone is observing the network, they will see that payment flow. If you host your node and your private wallet on it, it's complicated. If you are the only person hosting your wallet there, then the anonimity set is 1 so you are tracked. If many people are hosting their wallet there, the anonimity set is better but you have to trust that node (which if it's your node it's fine, but the others have to trust you) (this reminds me of vpn services lol)
VS, [13.09.21 11:40] [In reply to Testing Tester] basically some node -> @nitesh_btc -> LOOP and then letting the loop channel close (or rebalanced for lower if possible) The fees for forwarding to loop is 5000 as I understand from the discussion.
Testing Tester, [13.09.21 11:46] [In reply to VS] follow up question is: why "some node" would be fool enough to pay nitesh when they can just open a channel to loop directly?
VS, [13.09.21 11:50] [In reply to Testing Tester] some node is not paying @nitesh_btc most likely Nitesh is paying to rebalance that "some node" to push sats to remote at say X which when flow to LOOP pay Y. As long as Y remains > X there is some play left.
VS, [13.09.21 11:51] yes somenode can open to LOOP directly (with 16777215 minimum) and undercut Nitesh completely. It can happen slowly and not overnight because not everyone is still with 16M Sats on their on-chain wallet.
VS, [13.09.21 11:52] in long run the market grinds down the arbitrage
VS, [13.09.21 11:53] but if you look at it you deposit to bank at 0.1% bank lends you on credit card at 18% you can always ask -> why not someone starts lending at 5% or 10% to remove the credit card business - well that started peer to peer lending business BUT it has still not killed matercard or visa.
Things take a long time to change
VS, [13.09.21 11:54] the beauty of lightning is that it makes the field level - anyone can do it (just like anyone can open channel to LOOP and charge wahtever they want (1000 if they want to undercut Nitesh or 7000 if they want more return than Nitesh)
03d1e805c, [18.09.21 18:11] usually when i need to get some sats to open a channel i send them to my umbrel wallet from various sources at 1 sat/vbyte and then during channel open with ride the lightning i check to use unconfirmed utxo and then decide the final fee rate of lets say 3 sat/vbyte so according to CPFP the effctive fee rate of all the transactions involved becomes something in the middle weighted by their size, bunch of 1 sat/vbyte and 1x 3 sat/vbyte, that's how CPFP works
03d1e805c, [18.09.21 18:12] miners will now want to include those 1 sat/vbyte tx more than regular 1 sat/vbyte bc they know there's already a tx that will spend higher fee rate if they add those in, so they are worth more to them and 3 sat/vbyte is worth less bc it requires some underpriced tx to be added first
Testing Tester, [19.09.21 09:05] [In reply to 03d1e805c] is that still vanilla bos telegram bot? does it even tell you how often reconnect is triggered?
03d1e805c, [19.09.21 09:23] [In reply to Testing Tester] it's same bot but i'm just using same credentials bos uses to send messages through it
03d1e805c, [19.09.21 09:25] [In reply to Testing Tester] this is the easiest way to do it in js. token is the long string it gave you at first and chat_id is the short number it gave you later that i think is basically your user id so it knows to talk to you
03d1e805c, [19.09.21 09:30] [In reply to Testing Tester] i set it up so that if 1/3rd of peers disconnect it will try restarting tor and makes most sense to check that right after bos reconnect so I know that didn't solve it
Testing Tester, [19.09.21 09:32] I suppose you run the peers disconnection check script as root, right?
03d1e805c, [19.09.21 09:39] [In reply to Testing Tester] yeah i wanted to isolate the root script w/ 0 dependencies from the rest so i just made a sudo -b background process that checks if request to reset tor file appeared, which it tries and removes request file
03d1e805c, [19.09.21 09:41] [In reply to Testing Tester] [ Photo ] found out sudo -b is better than & for this bc i can put give it permission when launching it directly w/o saving "password" anywhere, then immidiately remove it via sudo -k for the rest of stuff I'm launching
Testing Tester, [19.09.21 09:46] [In reply to 03d1e805c] didn't know about -b switch on sudo but that's useful, I used a cron each minute to reboot system on telegram command on other raspis after inotifywait stopped working, but I guess also nohup or & might work on that script of yours, why did you choose to go with -b?
03d1e805c, [19.09.21 09:48] [In reply to Testing Tester] figured i might be updating stuff a lot, i have it set to terminate every time i relaunch by publishing a timestamp so any older timestamps end their processes
Full Throdl Hodl, [20.09.21 01:46] What are the risks associated with watchtower pairing? Does the other person get my channel backups, and is that dangerous? Or is the only significant risk that they could distribute my URI? Thanks in advance
VS, [20.09.21 07:51] [In reply to Full Throdl Hodl] No they get an encrypted revoke key which can only be decrypted when a commit transaction is published onchain. Once decrypting the revoke key watchtower checks if the revoke key is not revoked (ie channel state is old and revole key is newer). If that is the case, revoke key is used to publish justice transaction.
So in short watch tower can make sense of what it receives from you ONLY after it has received a commit transaction on chain related to your channe force close.
Until then it is gibberish for them.
El Profesor, [20.09.21 16:35] [In reply to VS] thanks! is there a --dry commands to do tests? I would like to avoid messing something up 😅 sorry, its my first time using bos for rebalancing
so for example, would something like this work out
bos rebalance --in channelwithlocal --channelwithhigherremore
VS, [20.09.21 16:39] [In reply to El Profesor] -- max-fee 1 it will act like a dry run
Testing Tester, [23.09.21 14:32] if I want to analyze how fees influence forwarding of a channel, is it useful to attach to the data the channel where the route came from?
Testing Tester, [23.09.21 14:33] I think not, but maybe you have wiser input
openoms, [23.09.21 14:34] [In reply to Testing Tester] yes, since you can only set outgoing fees, but what you are selling is the inbound capacity
openoms, [23.09.21 14:35] forwarding does equally depend on the incoming and outgoing channel
Testing Tester, [23.09.21 14:36] [In reply to openoms] noted, and how would that influence my decisions in the future? like, considering that incoming channel a good one, so I don't close it?
openoms, [23.09.21 14:37] [In reply to Testing Tester] would need create more incoming from that direction when exhausted
openoms, [23.09.21 14:38] use it as --out in a bos rebalance maybe with higher fee limits
[VS] it is easier to build a base first before - before worrying about centrality and hopness. generally keep 30-40% capital for big chunky channels possibly to main routes (which you would discover only after watching the flow), 30-40% for swaps/balanced channels you open in commuinity which gives you one inbound to one outbound with well connected nodes (you can do rings/triangle etc). Keep 20-30% for building the community/fun projects/helping/mentoring beginners nodes
Hollin Calloway, [9/24/21, 1:41 AM] I really need a script to rebalance out of channels where I have a ton of local balance
Hollin Calloway, [9/24/21, 1:41 AM] Scan all the possible routes to other channels that have more remote balance and picks ones where it is less expensive to rebalance
kp, [9/24/21, 1:54 AM] [In reply to Hollin Calloway]
- /15 * * * * /usr/bin/bos rebalance --in NodeA --in-target-outbound=capacity*0.95 --max-fee-rate=NNN --avoid badnode --minutes 10
kp, [9/24/21, 1:54 AM] put that in cronjob...it will run every 15 mins...and bring liquidity local for NodeA......h/t @nitesh_btc
Umbrella, [9/24/21, 1:55 AM] kp, let me ask a clarifying question. There's no --out because it's picked by bos?
kp, [9/24/21, 1:56 AM] [In reply to Umbrella] yes
kp, [9/24/21, 1:57 AM] [In reply to kp] You can replace --in with --out if you want to make everything remote.
- /15 * * * * /usr/bin/bos rebalance --out NodeA --out-target-inbound=capacity*0.95 --max-fee-rate=NNN --avoid badnode --minutes 10
Umbrella, [9/24/21, 1:57 AM] The --in-target-outbound=capacity*0.95, is that testing for the ratio it has now to gate execution?
Umbrella, [9/24/21, 1:57 AM] or is that setting a target? if so, why not 50:50
kp, [9/24/21, 1:58 AM] [In reply to Umbrella] Yes...it will execute for local liquidity less than 95% of the total channel capacity.
Umbrella, [9/24/21, 1:59 AM] excellent. I'm assuming we replace badnode with the pubkey of a node we want to avoid and minutes is how long to spend searching before giving up?
kp, [9/24/21, 2:00 AM] [In reply to Umbrella] yes....in my case "badnode" is a tag in bos....there is a list of nodes that it will avoid...I can't tell you that list 😜
Umbrella, [9/24/21, 2:03 AM] understood, but one can add a list of bad nodes to bos dir? What sort of criteria would one want in a bad node? high fees? no outbound?
Nitesh ₿⚡️, [9/24/21, 2:04 AM] [In reply to kp] Is it picking up the outpeer automatically? I’m not sure how good of a job is does in auto picking the out peer. What if it picks a peer you don’t want it to pick?
Nitesh ₿⚡️, [9/24/21, 2:05 AM] [In reply to Umbrella] Too much liquidity in one direction, maintains bad peers, very small channel capacity.
kp, [9/24/21, 2:05 AM] [In reply to Nitesh ₿⚡️] goes into avoid
Umbrella, [9/24/21, 2:06 AM] And, kp, one more question, unless someone sparks another with theirs, so this cronjob is really good only if we have a target node in mind rather than a list of 5 channels with 100% outgoing and all stuck. We'd do them one at a time, right?
kp, [9/24/21, 2:06 AM] [In reply to Umbrella] No outbound....high fees....some nodes out in there that you may not be connected that shows up on the paths like ln2me, 1ml, etc.
Hollin Calloway, [9/24/21, 2:08 AM] [In reply to kp] Can you set it to avoid nodes where you already have a lot of remote balance? Also is there a wiki that shows how to set a cron job for this?
Nitesh ₿⚡️, [9/24/21, 2:08 AM] [In reply to Nitesh ₿⚡️] Example of all these, 1ml.com.
kp, [9/24/21, 2:09 AM] [In reply to Hollin Calloway] yes....you can add --avoid NodeA --avoid NodeB etc.
kp, [9/24/21, 2:13 AM] [In reply to Umbrella] It goes back to the 1st question you have to ask.....why rebalance? rule-1 for me is I do not rebalance except the busy 2-3 channels that needs to be kept fed.......so pick the top 1-2 1st and then see if this with fees strategy improves the node forwards....and then get fancier....my opinion
Seems like turning off tor.streamisolation yesterday may have done the trick with the mysterious "Channel Stability" health check. Or, it was just coincidence. Probably just coincidence.
Dirk Krienbühl 🇨🇭, [28.09.21 16:02] [In reply to Testing Tester] OK, I seems that it's fixed and I get a proper validating json by now, as I added the following:
from google.protobuf.json_format import MessageToJson
and then converted the gRPC response using:
response_json = MessageToJson(response)
Now I can normalize the json into a panda dataframe using:
json_data = json.loads(response_json) df = json_normalize(json_data, 'payments')
Testing Tester, [9/29/21, 12:16 AM] mainnet node? I remember Richard reporting lnd logs to be extremely not helping in this case
Richard Safier (xenonfun), [9/29/21, 12:18 AM] [In reply to Testing Tester] yeah sometimes useful sometimes now, a lot of things are not as detailed as you might want . It helps to override default logging levels, but still a lot of the time just isn't great
Richard Safier (xenonfun), [9/29/21, 12:19 AM] debuglevel=CNCT=debug,CRTR=debug,HSWC=debug,NTFN=debug,RPCS=debug
VS, [29.09.21 22:30] [In reply to Martini] Timelock delta applies to htlc you accept Max cltv expiry applies when you are creating a payment route.
So a sender adds up timelock delta for all the peers in path and that must be less than max cltv expiry or else that path is not chosen.
Gridflare, [10/9/21, 5:26 AM] [In reply to 03d1e805c] Take channel ID and shift 40 bits to the right. That gets you the block height.
Nitesh ₿⚡️, [10/9/21, 5:53 AM] So the block height it gives is the first confirmation or the opening confirmation? I’m guessing it has to be opening right?
Gridflare, [10/9/21, 6:00 AM] First, because that's what you need to locate the tx
°°°°TRANSLATION: Divide channel ID by (2^40) to get blockheight
Nitesh ₿⚡️, [14.10.21 15:59] if we run bos or any gRPC scripts remotely (not the machine running LND) which basically connects to our node and runs the scripts using hostname and grpc port, would it be faster because of the device we are running on or wouldn't make a difference because its still in a way connecting to our node?
Nitesh ₿⚡️, [14.10.21 16:16] AWS is really crying with 4 parallel rebalances. so was seeing what can be done
Nitesh ₿⚡️, [14.10.21 16:33] interesting, i am trying it out now. there is no cpu usage bump on aws. guess its working
Zap, [14.10.21 16:33] I bet you would see better performance doing that remotely. My guess is there aren't enough cores for the 4 parallel rebalance processes. whilst the grpc interface is efficient enough to handle that amount of parallel io
Nitesh ₿⚡️, [14.10.21 16:34] yeah looks like its working wayyyy better
Zap, [14.10.21 16:34] I run all my rebalances using rebalance-lnd from a remote machine
Zap, [14.10.21 16:34] will be migrating the rest off too, RTL, TH
Nitesh ₿⚡️, [14.10.21 16:35] yeah I never installed TH on AWS coz it'll get stuck as soon as i do npm install LOL guess can move everything to remote.
Nitesh ₿⚡️, [14.10.21 16:35] [In reply to Zap] I am doing bos right now. it works great
Nitesh ₿⚡️, [14.10.21 16:36] [In reply to Zap] yup. its easy to setup
Nitesh ₿⚡️, [14.10.21 16:38] install bos
on your machine that runs lnd do, bos credentials —-cleartext it'll give you all credentials you need.
on your remote machine, mkdir ~/.bos cd into it. mkdir somerandomnodename cd into it and create a credentials.json file and put those credentials in it.
Nitesh ₿⚡️, [14.10.21 16:38] and you're done
Ngu Technologies, [14.10.21 16:41] so being in aws, you have to expose your grpc port to your local ip for security?
Nitesh ₿⚡️, [14.10.21 16:41] [In reply to Zap] when you run any command you have to do bos peers --node=somerandomnodename
you can skip that by making it default node by export BOS_DEFAULT_SAVED_NODE=somerandomnodename
Nitesh ₿⚡️, [14.10.21 16:42] [In reply to Ngu Technologies] yeah, i only set it to my local ip
Zap, [14.10.21 16:42] yes and that exposes in on that interface rather than just the default which I think is 0.0.0.0
Zap, [14.10.21 16:42] which means local devices can access the socket!
Zap, [14.10.21 16:43] so needs to be secured as it should be with tls.cert and macaroon claims
Ngu Technologies, [14.10.21 16:43] well guess you put in lnd.conf to only listen to remote ip, put on the ufw firewall and also on the security group lol
Ngu Technologies, [10/20/21, 4:28 AM] coingate has been an excellent peer and opennode seemed ok but the channel got closed and I did not get to reopening to try again
Full Throdl Hodl, [10/20/21, 5:24 AM] [In reply to Ngu Technologies] Coingate pushes liquidity back to you right?
Elwood the HODL ₿arbarian, [10/20/21, 5:25 AM] [In reply to Full Throdl Hodl] Does for me
Ngu Technologies, [10/20/21, 5:26 AM] coingate is bi-directional for me but a bit more towards inbound flow yes
Elwood the HODL ₿arbarian, [10/20/21, 5:27 AM] Boltz seems to push liquidity into local stronger than Coingate though
Elwood the HODL ₿arbarian, [10/20/21, 5:27 AM] For my node
Ngu Technologies, [10/20/21, 5:28 AM] Id agree, Bolts mostly inbound flows
kp, [10/20/21, 5:31 AM] [In reply to Ngu Technologies] Both Coingate and Boltz are bidirectional for me. Boltz more leaning to local Incoming.
Ngu Technologies, [10/20/21, 5:33 AM] indeed, those 2 are really good peers
Ngu Technologies, [10/20/21, 5:34 AM] outbound flow, I could say River is pretty good for that
kp, [10/20/21, 6:08 AM] Can you share the pubkey or Amboss link to their node. I have been thinking about it considering they are making a push into Lightning business. I believe they are selective about channels.
Ngu Technologies, [10/20/21, 6:09 AM] 03037dc08e9ac63b82581f79b662a4d0ceca8a8ca162b1af3551595b8f2d97b70a
kp, [10/20/21, 6:15 AM] So local to remote Outbound flow like WOS - right?
Ngu Technologies, [10/20/21, 6:16 AM] yeah, WoS does flow in somewhat for me but River flows in zero, rebalance only
03d1e805c, [25.10.21 10:42] you can do bos peers --offline
03d1e805c, [25.10.21 10:42] or bos reconnect will check and try fixing it first
03d1e805c, [25.10.21 10:43] I need to add exponential backoff to that reset % so doesn't get stuck in a loop tbh
BitcoinRevolution Advanced Node Security
Use Hardware-Based SSH keys
Logging in with a password or having your SSH keys on your hard drive is not secure. Both can easily be stolen if your computer is compromised. It is significantly more secure to log in with only an SSH key that even you have never been able to read the private key for. The hardware devices can also be configured to require physical touch for authentication, which is a massive security improvement.
1. Generate a hardware-based SSH key.
Check if your computer has a secure element. If it does, you may be able to use that. For Mac, you can use: https://github.com/maxgoedjen/secretive
The Ledger hardware wallet has a secure element and can generate SSH keys within it: https://blog.ledger.com/ssh/
You can buy a ubikey and generate SSH keys on that. It's also very good for two factor authentication codes: https://developers.yubico.com/SSH/
2. Once the hardware-based SSH keys are generated, you'll need to add the public key as the first line in ~/.ssh/authorized_keys on your node. ie: echo "PUBLIC KEY GOES HERE" >> ~/.ssh/authorized_keys
Use SSH-Tunneling For the Web Interface
Being able to log in to Umbrel, Thunderhub, RTL, etc. with just a password on the local network is not secure, especially since those websites don't use HTTPS on the local network. Using only a password is not secure. Require hardware-based SSH key log in to access those interfaces until Umbrel adds two factor authentication to the interface.
I'd recommend not using the Tor URLs for accessing the interfaces for Umbrel or apps. Most of the apps use a default password that's publicly known. If the tor URL is stolen, they will have access to your funds. Even if you change the password, just password-based authentication is not secure as even the password can be stolen using a simple key logger.
1. Create a SOCKS5 proxy using SSH: ssh -D 8123 -f -C -q -N email@example.com
2. Set up your browser or computer to use the proxy. In Firefox, you can search the settings for the word "proxy" and then choose "manual proxy configuration". Enter "localhost" for the SOCKS5 Host and 8123 as the port. I use Brave for my normal browsing and Firefox for my node related things so that only those things are proxied through my node.
3. Configure your network firewall or the firewall on your Umbrel (using iptables) to block the outbound ports for Umbrel, RTL, Thunderhub, etc. I don't have the iptables commands for this because I use an external firewall.
# # Run stream-lnd-htlcs as a service # [Unit] Description=Stream LND HTLCS Service After=lnd.service Wants=network-online.target [Service] Type=simple ExecStart=python /path/to/stream-lnd-htlcs --options KillMode=process TimeoutStartSec=60 TimeoutStopSec=60 Restart=always RestartSec=60 User=YOURUSER Group=YOURGROUP [Install] WantedBy=multi-user.target
#!/bin/bash IGNORE="YOURNODEALIAS" INPUT=/tmp/htlc-stream.json #whatever the path to your json is if [ -f "$INPUT" ]; then c_total=`grep -v $IGNORE $INPUT | grep FORWARD | wc -l` a_tcf=`grep -v $IGNORE $INPUT | grep "FORWARD.*INSUFFICIENT_BALANCE" | wc -l` b_fee=`grep -v $IGNORE $INPUT | grep "FORWARD.*FEE_INSUFFICIENT" | wc -l` e_htlc=`grep -v $IGNORE $INPUT | grep "FORWARD.*HTLC_EXCEEDS_MAX" | wc -l` d_settle=`grep -v $IGNORE $INPUT | grep "FORWARD.*settle_event" | wc -l` printf "%(%Y-%m-%d)T %(%T)T\n" printf " %08s %08s %08s %08s %08s" "total" "settled" "ins.bal." "ins.fee" "ins.htlc" printf "\n" printf " %08d %08d %08d %08d %08d" $((10#$c_total)) $((10#$d_settle)) $((10#$a_tcf)) $((10#$b_fee)) $((10#$e_htlc)) printf "\n" rm $INPUT else printf "No events recorded since last time\n" fi
Indomitus, [In reply to kp] Since I implemented @LN_03d1e805c maxbackoff=5m, connections have been much more solid. Coincidence?. Perhaps who knows Default is 1h. You basically add this line under applications options
#######root crontab #simple smart checks and btrfs stats everyday 0 0 * * * /home/bitcoin/smartmonitor sda 1 0 * * * /home/bitcoin/smartmonitor sdb 2 0 * * * btrfs device stats / > /tmp/btrfs-stats-root 3 0 * * * btrfs device stats /mnt/btc > /tmp/btrfs-stats-btc #short smart self tests every 2 weeks ##starts test on sda 10 0 14,28 * * smartctl -d sat -t short /dev/sda ##reads results for sda 15 0 14,28 * * smartctl -d sat -l selftest /dev/sda > /tmp/smartshort-sda ##starts test on sdb 16 0 14,28 * * smartctl -d sat -t short /dev/sdb ##reads results for sdb 21 0 14,28 * * smartctl -d sat -l selftest /dev/sdb > /tmp/smartshort-sdb #monthly btrfs scrub ## rootfs start scrub 0 2 3 * * btrfs scrub start / ## rootfs check scrub 40 minutes later 40 2 3 * * btrfs scrub status / > /tmp/scrub-root ## bitcoin start scrub 41 2 3 * * btrfs scrub start /mnt/btc/ ## bitcoin check scrub 3 hours later 41 5 3 * * btrfs scrub status /mnt/btc/ > /tmp/scrub-bitcoin #####smartmonitor #!/bin/bash date > /tmp/smartmon- smartctl -d sat -H /dev/ | grep -i overall >> /tmp/smartmon- smartctl -d sat -a /dev/ | grep -i reallocated >> /tmp/smartmon- smartctl -d sat -a /dev/ | grep -i reported >> /tmp/smartmon- smartctl -d sat -l error /dev/ >> /tmp/smartmon-
Btw if you’re testing your SCB recovery. You should also test with spinning up a neutrino with docker or something and do a recovery from there. A lot of times SCB recovery on your node isn’t possible coz of disk failures and if disk fails, you don’t have your core and you have to sync from start which is a pain in the ass unless you keep a spare core synced somewhere.
Nitesh ₿⚡️, [In reply to M.] Ah ok. Yeah it affects your path finding with rebalances and payments.
Nitesh ₿⚡️, [routerrpc]
- Set default chance of a hop success
- Start to ignore nodes if they return many failures (set to 1 to turn off)
- Set minimum desired savings of trying a cheaper path
- Set the number of historical routing records
- Set the min confidence in a path worth trying
- Set the time to forget past routing failures
Move lnd installation with postgres enabled to different system
- let's assume postgres database name is `bitcoin`
`pg_dump bitcoin > /path/to/bitcoin_database.dump` (backup lnd folder)
...copy lnd folder backup and postgres dump to second system...
(restore lnd folder)
CREATE DATABASE bitcoin; -- mind the ending ;
Update `ufw` for each service you add:
`sudo ufw allow <port.no>`
Stop lnd from filling up /var/log
Need to add StandardOutput=null in service definition file right under the ExecStart line, and those logs won't be spammed anymore.
A minor inconvenience is that I cannot use journalctl -f -u lnd anymore, but I need to run tail -f /home/bitcoin/lnd/logs/blahblah
Move posstgres data folder
(make sure all folders above that have at least 755 permission, and all below are chowned to postgres:postgres)
Multiple postgres instances on raspi/debian:
Replication on postgres
>Bitcoin txn size breakdown for single sig:
- Base tx, one input + one output = 110vB
- Each additional input, +67.5vB
- Each additional output, +31vB
osito: just been hit by a segmentation fault in LND, no idea what went wrong tho. lnd restarted fine it seems. is this worth opening an issue on lnd?
Augustus Baggins: Likely https://github.com/lightningnetwork/lnd/issues/6061#issuecomment-995593368 - will be fixed in 0.14.2, but at the moment set routerrpc.maxmchistory=0
bos limit-forwarding --disable-forwards
...to stop forwards before shutting down
Full Throdl Hodl, [In reply to Full Throdl Hodl] Ok I know how to prevent further htlcs, but what allows me to broadcast to my peers that I will be offline for a few days?
Also I’m going to have a force close mature while I’m offline. Do I need to be concerned about that?
Nitesh ₿⚡️, [In reply to Full Throdl Hodl] bos advertise —max-hops=0
Nitesh ₿⚡️, That’ll advertise only to peers.
Nitesh ₿⚡️, With the message flag ofcourse. —message=“I’m going on vacation, don’t force close on me”
check pending htlcs: lncli listchannels | jq '. | .pending_htlcs '
if some htlc is near its expiration_height (like let's say 5 blocks), it could be considered unsafe to restart.
2FA Two factor authentication https://www.digitalocean.com/community/tutorials/how-to-set-up-multi-factor-authentication-for-ssh-on-ubuntu-18-04