aboutsummaryrefslogtreecommitdiff
path: root/external
Commit message (Collapse)AuthorAgeFilesLines
...
* updated in-source lmdbRiccardo Spagni2015-07-1610-95/+171
|
* open() flag O_DSYNC isn't on BSD, use O_SYNCThomas Winget2015-07-162-0/+8
| | | | | | If the detected OS is FreeBSD, tell LMDB to compile with MDB_DSYNC=O_SYNC instead of the default O_DSYNC, as BSD does not implement this flag.
* ** CHANGES ARE EXPERIMENTAL (FOR TESTING ONLY)NoodleDoodleNoodleDoodleNoodleDoodleNoo2015-07-151-20/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bockchain: 1. Optim: Multi-thread long-hash computation when encountering groups of blocks. 2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible. 3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible. 4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks. 5. Optim: Multi-thread signature computation whenever possible. 6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD) 7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???). 8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads). Berkeley-DB: 1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc). 2. Fix: Unable to pop blocks on reorganize due to transaction errors. 3. Patch: Large number of transaction aborts when running multi-threaded bulk queries. 4. Patch: Insufficient locks error when running full sync. 5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation. 6. Optim: Add bulk queries to get output global indices. 7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3) 8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key 9. Optim: Added thread-safe buffers used when multi-threading bulk queries. 10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details) 11. Mod: Added checkpoint thread and auto-remove-logs option. 12. *Now usable on 32-bit systems like RPI2. LMDB: 1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect) 2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3) 3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key 4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details) 5. Mod: Auto resize to +1GB instead of multiplier x1.5 ETC: 1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete. 2. Fix: 32-bit saturation bug when computing next difficulty on large blocks. [PENDING ISSUES] 1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization. This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD. 2. Berkeley db, possible bug "unable to allocate memory". TBD. [NEW OPTIONS] (*Currently all enabled for testing purposes) 1. --fast-block-sync arg=[0:1] (default: 1) a. 0 = Compute long hash per block (may take a while depending on CPU) b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence) 2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000) a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions. b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache. Fast - Write meta-data but defer data flush. Fastest - Defer meta-data and data flush. Sync - Flush data after nblocks_per_sync and wait. Async - Flush data after nblocks_per_sync but do not wait for the operation to finish. 3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower) Max number of threads to use when computing long-hash in groups. 4. --show-time-stats arg=[0:1] (default: 1) Show benchmark related time stats. 5. --db-auto-remove-logs arg=[0:1] (default: 1) For berkeley-db only. Auto remove logs if enabled. **Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version. At the moment, you need a full resync to use this optimized version. [PERFORMANCE COMPARISON] **Some figures are approximations only. Using a baseline machine of an i7-2600K+SSD+(with full pow computation): 1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain. 2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain. 3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain. Averate procesing times (with full pow computation): lmdb-optimized: 1. tx_ave = 2.5 ms / tx 2. block_ave = 5.87 ms / block memory-official-repo: 1. tx_ave = 8.85 ms / tx 2. block_ave = 19.68 ms / block lmdb-official-repo (0f4a036437fd41a5498ee5e74e2422ea6177aa3e) 1. tx_ave = 47.8 ms / tx 2. block_ave = 64.2 ms / block **Note: The following data denotes processing times only (does not include p2p download time) lmdb-optimized processing times (with full pow computation): 1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000). 2. Laptop, Dual-core / 4-threads U4200 (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000). 3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000). lmdb-optimized processing times (with per-block-checkpoint) 1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000). berkeley-db optimized processing times (with full pow computation) 1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000). 2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000). berkeley-db optimized processing times (with per-block-checkpoint) 1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
* re-add Windows DNS bug fix, per c0de96f8bd2f64884255f9b0b61a4775ada5e3e6Riccardo Spagni2015-06-021-1/+0
|
* update libunboundRiccardo Spagni2015-05-3159-2463/+4447
|
* Require BerkeleyDB to be installed (for now) if building non-staticThomas Winget2015-04-221-7/+2
|
* update lmdb64Riccardo Spagni2015-04-1422-365/+201
|
* Merge BlockchainDB into upstreamThomas Winget2015-04-0759-0/+32311
|\
| * Only compile BerkeleyDB as an option in non-staticThomas Winget2015-04-071-16/+18
| |
| * Merge upstream into blockchainThomas Winget2015-03-291-1/+1
| |\
| * \ Pull blockchain changes into berkeleydb branchThomas Winget2015-03-1759-36/+16093
| |\ \
| | * | Move db_drivers/ to external/Thomas Winget2015-03-1758-0/+32259
| | | | | | | | | | | | | | | | | | | | Also change LMDB Cmake variables to CACHE rather than upgrading them through several parent scopes.
| | * | fixed msys2 / mingw folders based on architecture, added license to unbound ↵Riccardo Spagni2015-03-171-0/+28
| | | | | | | | | | | | | | | | CMakeList as that is not part of standard Unbound
| | * | Revert "Moved db_drivers/ into external/ for consistency"Thomas Winget2015-03-1730-16221/+0
| | | | | | | | | | | | | | | | This reverts commit b21335642e75b35d3b178a754f4cdb2314989cd1.
| | * | Revert "Build fixed, goofed up some CMake"Thomas Winget2015-03-171-4/+0
| | | | | | | | | | | | | | | | This reverts commit 8b82f3c57ffdc4dc190f4755c61d74d8359c8c0f.
| * | | BerkeleyDB Blockchain building, not working yetThomas Winget2015-03-161-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Everything except actually *using* BlockchainBDB is wired up, but the db itself is not yet working. Some error about user mem not large enough. I think I know what this error means, but I can't determine the cause. Notes: BerkeleyDB does not allow 0-indexing in its recno type databases, so block numbers *in the database* will be 1-indexed. Modifications to indexing have been made as needed.
| * | | CMake wiring, minor cleanup, minor test additionThomas Winget2015-03-162-1/+28
| |/ / | | | | | | | | | | | | | | | Make Cmake things aware of BerkeleyDB and BlockchainBDB Make the BlockchainDB unit tests aware of BlockchainBDB
| * | Build fixed, goofed up some CMakeThomas Winget2015-03-101-0/+4
| | | | | | | | | | | | | | | | | | Forgot that CMake vars set to PARENT_SCOPE will still vanish if that parent scope goes...out of scope. LMDB vars elevated one more scope to compensate for moving db_drivers/ into external/
| * | Moved db_drivers/ into external/ for consistencyThomas Winget2015-03-0930-0/+16221
| | |
| * | Remove unused dependencyThomas Winget2015-01-0426-11043/+0
| | |
| * | Adding libglim as an external libraryThomas Winget2015-01-0426-0/+11043
| | | | | | | | | | | | | | | | | | | | | | | | | | | libglim is an Apache-licensed C++ wrapper for lmdb, and rather than rolling our own it seems prudent to use it. Note: lmdb is not included in it, and unless something happens as did with libunbound, should be installed via each OS' package manager or equivalent.
* | | updated unbound cmake for static buildsRiccardo Spagni2015-04-063-2/+34
| | |
* | | minor CMakeLists fix for Unbound, ldns -> sldnsRiccardo Spagni2015-04-051-7/+7
| | |
* | | reinstate accidentally removed CMakeListsRiccardo Spagni2015-04-041-0/+201
| | |
* | | Merge pull request #251Riccardo Spagni2015-04-021-1/+0
|\ \ \ | | | | | | | | | | | | c0de96f Fixed DNS resolution bug in Windows (Thomas Winget)
| * | | Fixed DNS resolution bug in WindowsThomas Winget2015-04-021-1/+0
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to a bug in unbound, we were passing a string containing a null character to ub_ctx_resolvconf and ub_ctx_hosts rather than a NULL pointer. On *nix this wasn't causing headache, but on Windows this was causing unbound to not correctly load DNS settings from the OS. Note on the bug: in a Windows-specific code branch in the function ub_ctx_hosts(), if the hosts file specified was a NULL pointer, a call to getenv() was stored in a local char* and later freed. This is incorrect, as we do not own that data, and caused the program to crash.
* / | update unbound from upstreamRiccardo Spagni2015-04-02155-3409/+5451
|/ /
* / fix cmake miniupnpc typomeshpoint2015-03-261-1/+1
|/ | | which matters on case-sensitive filesystems
* year updated in licenseRiccardo Spagni2015-01-0214-3/+3
|
* update unbound from upstreamRiccardo Spagni2014-12-0472-2655/+1261
|
* moved rapidjson to external folder, fixed CMakeRiccardo Spagni2014-12-0124-0/+8020
|
* mingw: copy required libraries to the build treeBen Boeckel2014-11-181-0/+16
| | | | | These are found as shared libraries and need to be copied so that PATH manipulation isn't necessary outside of an msys shell.
* miniupnpc: bump the _POSIX_C_SOURCE feature macroBen Boeckel2014-11-101-1/+1
| | | | FreeBSD doesn't expose IPv6 structures without this.
* unbound: fix getaddrinfo detection for 32-bit windowsBen Boeckel2014-10-241-1/+11
| | | | | | | | | | On Windows, getaddrinfo is part of the Windows API and as such is __stdcall, not __cdecl, so check_function_exists fails because the declaration doesn't match the mangling __stdcall has. Instead, use a header to include the symbol as declared on the system and use check_symbol_exists instead. Tested-By: greatwolf on IRC
* unbound: fix type checkingBen Boeckel2014-10-241-2/+2
|
* unbound: plumb the libdir upBen Boeckel2014-10-242-1/+6
| | | | | This is necessary for static builds where the linking is passed around to dependent targets, but the library is lost.
* cmake: support 2.8.7Ben Boeckel2014-10-241-4/+4
| | | | | | Older versions of CMake support LINK_{PUBLIC,PRIVATE} while newer versions prefer PUBLIC and PRIVATE instead, but still support the LINK_ prefix.
* cmake: remove scream-make (all-caps functions)Ben Boeckel2014-10-231-15/+15
|
* unbound: import cmake build systemBen Boeckel2014-10-234-82/+1428
|
* miniupnpc: clean up build systemBen Boeckel2014-10-231-65/+5
| | | | | | | | | Changes: - remove amiga and solaris conditionals - remove -fPIC hackery (use POSITION_INDEPENDENT_CODE instead) - remove tests - retab
* cmake: minor cleanups (indentation and typos)Ben Boeckel2014-10-231-0/+0
|
* cmake: remove configuration variablesBen Boeckel2014-10-231-11/+0
| | | | | | | CMAKE_BUILD_TYPE is meant for single-config build tools (e.g., make and ninja) while CMAKE_CONFIGURATION_TYPES is meant for multi-config build tools (e.g., Xcode and Visual Studio). They should not be mixed or manually set.
* cmake: factor out error messagesBen Boeckel2014-10-231-3/+3
| | | | | Instead of using BoldRed and ColourReset everywhere, wrap it up in a function.
* miniupnpc: clear out else/endfoo command argumentsBen Boeckel2014-10-231-15/+15
|
* fixed miniupnpc dynamic targetRiccardo Spagni2014-10-061-2/+4
|
* fix for mingw not playing nicely with libunbound configure, fix for ↵Riccardo Spagni2014-10-062-1/+7
| | | | correctly finding static libs on various operating systems
* set the winsock variable differently on WindowsRiccardo Spagni2014-10-061-1/+2
|
* fixed configure_command for libunbound under mingwRiccardo Spagni2014-10-061-1/+1
|
* turns out mingw does actually produce .a libs and not .dll.aRiccardo Spagni2014-10-061-3/+1
|
* quieten CMake when it can't find packagesRiccardo Spagni2014-10-061-3/+3
|
generated by cgit v1.2.3 (git 2.51.0) at 2026-08-17 02:45:07 +0000