latest version Go Report Card Travis branch license

    abc banner image


    ABC is a command-line tool to interact with It can also serve as a swiss army knife to import data from any popular data source (Postgres, SQL, Mongo) to ElasticSearch. This feature works with minimum configuration and is totally automatic. In certain sources like Postgres and Mongo, you can even keep the database and ElasticSearch cluster in sync such that any change from source gets added in destination as well.

    1. Intro
    2. Key Benefits
    3. Getting Started
    4. Features
      1. Appbase features
      2. Importer features
    5. Development setup
      1. Local Setup
      2. Docker Setup
      3. Build Variants
    6. ABC Resources
      1. Contributing to ABC
      2. Licensing

    1. Intro

    ABC consists of two parts.

    1. Appbase module
    2. Import module (closed source)

    To get the list of all commands supported by ABC, use -

    abc --help

    2. Key Benefits

    ABC comes with a lots of benefits over any other traditional solution to the same problem. Some of the key points are as follows -

    • Whether your data resides in Postgres or a JSON file or MongoDB or in all three places, abc can index the data into Elasticsearch. Besides these, it also supports CSV, MySQL, SQLServer, Kafka and Elasticsearch itself to an Elasticsearch index.
    • It can keep the Elasticsearch index synced in realtime with the data source. (Note: Currently only supported for MongoDB and Postgres)
    • abc import is a single line CLI command that allows doing all of the above. It doesn’t require any external dependencies, takes zero lines of code configuration, and runs as an isolated process with a minimal resource footprint.
    • abc also supports configurable user defined transformations for advanced uses to map data types, columns or transform the data itself before it gets indexed into Elasticsearch.

    3. Getting Started

    ABC can be downloaded as an executable as well as through a Docker image.

    Using Executable

    Download abc's executable from releases for your platform and preferrably put it in a PATH directory. The access it as -

    > abc

    You should see a list of commands that abc supports. Try logging in for example.

    Using Docker

    To use the Docker image, pull it as

    docker pull appbaseio/abc

    Then create the volume to store config files across containers.

    docker volume create --name abc

    Finally you should be able to use abc

    docker run -i --rm -v abc:/root appbaseio/abc

    This command may look too long to you. We can create an alias to make things better.

    # create alias
    alias abc='docker run -i --rm -v abc:/root appbaseio/abc'
    # run a command
    abc login google

    4. Features

    ABC's features can be broadly categorized into 2 components.

    1. Appbase features
    2. Importer features

    4.1 Appbase features

    Appbase features allows you to control your account using ABC. You can see them under the Appbase heading in the list of commands.

      login     login into
      user      get user details
      apps      display user apps
      app       display app details
      create    create app
      delete    delete app
      logout    logout session
      import    import data from various sources into appbase app

    You can look over help for each of these commands using the --help switch. Alternatively we have detailed docs for them at docs/appbase folder.

    abc login --help


    # display all commands
    # login into system
    abc login google
    # get user details
    abc user
    # get list of apps
    abc apps
    # get details of an app
    abc app MyAppName
    # delete that app
    abc delete MyAppName
    # create it again
    abc create MyAppName
    # view its metrics. It will be 0 as it is a new app
    # here we are using AppID. We can use AppName too.
    abc app -m 2489

    4.2 Importer features

    ABC allows the user to configure a number of data adaptors as sources or sinks. These can be databases, files or other resources. Data is read from the sources, converted into a message format, and then send down to the sink where the message is converted into a writable format for its destination. The user can also create data transformations in JavaScript which can sit between the source and sink and manipulate or filter the message flow.

    Adaptors may be able to track changes as they happen in source data. This "tail" capability allows a ABC to stay running and keep the sinks in sync. For more details on adaptors, see Import docs.

    5. Development

    ABC can be built locally via the traditional go build or by building a Docker image.

    5.1 Local Setup

    You can install ABC by building it locally and then moving the executable to anywhere you like.

    To build it, you will require Go 1.8 or above installed on your system.

    go get # alternatively, clone the repo in the `$GOPATH/src/` dir
    cd $GOPATH/src/
    go build -tags 'oss' ./cmd/abc/...
    ./abc --help  # voila, you just built abc from source!

    Note - You might be wondering what is the tag oss doing there. That's covered in the section Build Variants.

    5.2 Docker Setup

    git clone
    cd abc
    docker build --build-arg ABC_BUILD=oss -t abc .
    docker volume create --name abc

    Volume is used to store abc config files across containers. Now abc can be ran through Docker like in the following example which starts google login.

    docker run -i --rm -v abc:/root abc login google

    Some more examples

    # setting alias for easy usage
    alias abc='docker run -i --rm -v abc:/root abc'
    # using alias now :)
    abc user
    abc apps

    5.3 Build Variants

    The ABC project you see in this repository is not the complete project. works on a proprietary version of ABC using this project as the base. Hence we use the tag 'oss' to specify that this is an open source build. If you are curious, we use the tag '!oss' to make our private builds.

    How to know build variant from the executable?

    If you are not sure which build of abc you are using, you can run abc version and take note of the value under the VERSION header.

    For open source build, you will see

      ... (oss)

    For the proprietary builds, you will see

      ... (!oss)

    6. ABC Resources

    Checkout the docs folder for details on some ABC commands and topics.

    6.1 Contributing to ABC

    Want to help out with ABC? Great! There are instructions to get you started here.

    6.2 Licensing

    ABC's oss variant is licensed under the Apache 2.0 License. See LICENSE for full license text. ABC's !oss (read non-oss) variant which includes the abc import command and bundled in the binary is free to use while in beta.



    发行版本 31

    1.0.0 Beta Release 2


    贡献者 14



    • Go 98.4 %
    • Dockerfile 1.0 %
    • Shell 0.6 %