for storing batch keys and values in boltDBBatch.
NOTE: batch does not have to be safe for concurrent access. Delete may
be slow, but given it should not be used often, it's ok.
Fixes#3631
* libs/db: conditional compilation
For cleveldb: go build -tags cleveldb
For boltdb: go build -tags boltdb
Fixes#3611
* document db_backend param better
* remove deprecated LevelDBBackend
* update changelog
* add missing lines
* add new line
* fix TestRemoteDB
* add a line about boltdb tag
* Revert "remove deprecated LevelDBBackend"
This reverts commit 1aa85453f76605e0c4d967601bbe26240e668d51.
* make PR non breaking
* change DEPRECATED label format
https://stackoverflow.com/a/36360323/820520
Description:
add boltdb implement for db interface.
The boltdb is safe and high performence embedded KV database. It is based on B+tree and Mmap.
Now it is maintained by etcd-io org. https://github.com/etcd-io/bbolt
It is much better than LSM tree (levelDB) when RANDOM and SCAN read.
Replaced #3604
Refs #1835
Commits:
* add boltdb for storage
* update boltdb in gopkg.toml
* update bbolt in Gopkg.lock
* dep update Gopkg.lock
* add boltdb'bucket when create Boltdb
* some modifies
* address my own comments
* fix most of the Iterator tests for boltdb
* bolt does not support concurrent writes while iterating
* add WARNING word
* note that ReadOnly option is not supported
* fixes after Ismail's review
Co-Authored-By: melekes <anton.kalyaev@gmail.com>
* panic if View returns error
plus specify bucket in the top comment
* Remove deprecated PanicXXX functions from codebase
As per discussion over
[here](https://github.com/tendermint/tendermint/pull/3456#discussion_r278423492),
we need to remove these `PanicXXX` functions and eliminate our
dependence on them. In this PR, each and every `PanicXXX` function call
is replaced with a simple `panic` call.
* add a changelog entry
Fixes#3457
The topic of the issue is that : write a BlockRequest int requestsCh channel will create an timer at the same time that stop the peer 15s later if no block have been received . But pop a BlockRequest from requestsCh and send it out may delay more than 15s later. So that the peer will be stopped for error("send nothing to us").
Extracting requestsCh into its own goroutine can make sure that every BlockRequest been handled timely.
Instead of the requestsCh handling, we should probably pull the didProcessCh handling in a separate go routine since this is the one "starving" the other channel handlers. I believe the way it is right now, we still have issues with high delays in errorsCh handling that might cause sending requests to invalid/ disconnected peers.
Fixes: #3378
* Add stats to cleveldb implementation
* update changelog
* remote TODO
also
- sort keys
- preallocate memory
* fix const initializer []string literal is not a constant
* add test
* not related to linter: remove obsolete constants:
- `Insecure` and `Secure` and type `Security` are not used anywhere
* not related to linter: update example
- NewInsecure was deleted; change example to NewRemoteDB
* address: Binds to all network interfaces (gosec):
- bind to localhost instead of 0.0.0.0
- regenerate test key and cert for this purpose (was valid for ::) and
otherwise we would see:
transport: authentication handshake failed: x509: certificate is
valid for ::, not 127.0.0.1\"
(used https://github.com/google/keytransparency/blob/master/scripts/gen_server_keys.sh
to regenerate certs)
* use sha256 in tests instead of md5; time difference is negligible
* nolint usage of math/rand in test and add comment on its import
- crypto/rand is slower and we do not need sth more secure in tests
* enable linter in circle-ci
* another nolint math/rand in test
* replace another occurrence of md5
* consistent comment about importing math/rand
Why:
original fork is abandoned and not supported anymore.
Changes:
- LevelDB 1.19 (LevelDB and Snappy are both compiled and linked statically, so while you will not need them installed on your target machine, you should have a roughly compatible version of libstdc++.)
- snappy and lz4 libs included by default
* Enforce file permissions in case they've changed
* test behaviour for autofile
* use testify in tests and rename `fInf` to `fileInfo`
* return an error if file permissions have changed
- if we can't read the file, we'll still panic
* get rid of "github.com/pkg/errors" dependency
* address review comments:
- prefix instead of suffix
- add state to err and construct formatting in Error() method
* address review comments:
- move error to libs/errors
