started work to convert types to/from binary and json

ROADMAP.md

-EOS.IO Software Roadmap
------------------------
-
-This document outlines the development plan from a high level and will be updated as progress is
-made toward version 1.0. It should be noted that this roadmap applies only to the blockchain software
-and not to the other tools and utilities such as wallets and block explorers which will have their own
-teams and dedicated roadmaps once Phase 1 is complete..
-
-***Everything contained in this document is in draft form and subject to change at any time and provided for information purposes only.  
-block.one does not guarantee the accuracy of the information contained in this roadmap and the information is provided “as is” with no 
-representations or warranties, express or implied.***
-
-# Phase 1 - Minimal Viable Testing Environment - Summer 2017
-
-The goal of this phase is to establish the APIs that developers will require to start building and
-testing applications on EOS.IO. In order for developers to start testing their applications they 
-will require the following to be implemented:
-
-### Standalone Node  (Dan & Nathan)
-
-A standalone node operates a test blockchain and produces blocks while exposing an API. This node
-does not need to concern itself with any P2P networking code.
-
-### Native Contracts (Nathan)
-
-The EOS.IO software has a number of native contracts. These are contracts that manage the core
-operations of the blockchain and exist outside the Web Assembly interface. These contracts include:
-
-   1. @eos - manages EOS token transfers
-   2. @stake - manages locked EOS, voting, and Producer Election
-   3. @system - manages permissions, messages, and contact code updates
-
-### Virtual Machine API  (Dan)
-
-Contracts are compiled to WebAssembly (WASM) and WASM must interface with the blockchain via a defined
-API. This API is what developers depend upon to build applications and be relatively stable before
-developers can really start to build on EOS.
-
-### RPC Interface (Arhag, Nathan)
-
-A simple JSON RPC over HTTP interface will be provided that enables developers to broadcast transactions
-and query application state. This is critical for both publishing and interacting with test applications.
-
-### Command line Tools (Arhag)
-
-Command line tools facilitate integrating the RPC interface with developer build environments. 
-
-### Basic Developer Documentation (Josh)
-
-Documents that teach developers how to get started with building on EOS.IO blockchains. This includes
-documentations of the WASM API, RPC Interface, and Command Line Tools.
-
-
-# Phase 2 - Minimal Viable Test Network - Fall 2017
-
-Everything in Phase 1 assumes a trusted environment that only runs the developer's own code. Before a test
-network can be deployed several additional features need to be implemented and tested.
-
-### P2P Network Code (Phil)
-
-This is a plugin that is responsible for synchronizing the blockchain state between two standalone nodes.  
-
-### WASM Sanitation & CPU Sandboxing (Brian)
-
-The WASM code needs to be sanitized to check for non-deterministic behavior such as floating point operations and
-infinite loops.  
-
-### Resource Usage Tracking & Rate Limiting ( Arhag )
-
-To prevent abuse the resource monitoring and usage tracking rate limits users accoding to staked EOS.
-
-### Genesis Import Testing (DappHub)
-
-Tools need to be developed to export data from the EOS Token Distribution state and create a genesis configuration file. This will
-enable anyone participating in the Token Distribution to acquire some initial test EOS (TEOS) 
-
-### Interblockchain Communication (Nathan)
-
-This feature involves verifying the Merkle hashing of transactions is proper.
-
-# Phase 3 - Testing & Security Audits - Winter 2017, Spring 2018 
-
-During this phase the platform will undergo heavy testing with a focus on finding security issues and bug. At the end of Phase 3 version 1.0 will be tagged.
-
-### Develop Example Applications
-
-Example applications are critical to proving the platform provides the features required by real developers.
-
-### Bounties for Succesfully Attacking Network 
-
-Attacking the network with spam, virtual machine exploits, and bug crashes, and non-deterministic behavior will be a heavily involved process but
-necessary to ensure that version 1.0 is stable.
-
-### Language Support
-
-Adding support for additional langauges to be compiled to WASM: C++, Rust, etc.  
-
-### Documentation & Tutorials
-
-# Phase 4 - Parallel Optimization  Summer / Fall 2018
-
-After getting a stable 1.0 product released, we will move toward optimizing the code for parallel execution. 
-
-# Phase 5 - Cluster Implementation  The Future

libraries/chain/chain_controller.cpp

@@ -491,9 +491,6 @@ void chain_controller::_apply_block(const signed_block& next_block)
    create_block_summary(next_block);
    clear_expired_transactions();
 
-   if( !_node_property_object.debug_updates.empty() )
-      apply_debug_updates();
-
    // notify observers that the block has been applied
    // TODO: do this outside the write lock...? 
    applied_block( next_block ); //emit
@@ -735,22 +732,6 @@ bool chain_controller::before_last_checkpoint()const {
    return (_checkpoints.size() > 0) && (_checkpoints.rbegin()->first >= head_block_num());
 }
 
-/**
- *  This method dumps the state of the blockchain in a semi-human readable form for the
- *  purpose of tracking down funds and mismatches in currency allocation
- */
-void chain_controller::debug_dump() {
-}
-
-void debug_apply_update(chain_controller& db, const fc::variant_object& vo) {
-}
-
-void chain_controller::apply_debug_updates() {
-}
-
-void chain_controller::debug_update(const fc::variant_object& update) {
-}
-
 const global_property_object& chain_controller::get_global_properties()const {
    return _db.get<global_property_object>();
 }
@@ -849,17 +830,6 @@ void chain_controller::initialize_chain(chain_initializer_interface& starter)
 chain_controller::chain_controller(database& database, fork_database& fork_db, block_log& blocklog,
                                    chain_initializer_interface& starter, unique_ptr<chain_administration_interface> admin)
    : _db(database), _fork_db(fork_db), _block_log(blocklog), _admin(std::move(admin)) {
-      /*
-   static bool bound_apply = [](){
-      wrenpp::beginModule( "main" )
-         .bindClassReference<apply_context>( "ApplyContext" )
-            .bindFunction< decltype( &apply_context::get ), &apply_context::get >( false, "get(_)")
-            .bindFunction< decltype( &apply_context::set ), &apply_context::set >( false, "set(_,_)")
-         .endClass()
-      .endModule();
-      return true;
-   }();
-   */
 
    initialize_indexes();
    starter.register_types(*this, _db);
@@ -1114,4 +1084,60 @@ void chain_controller::set_apply_handler( const AccountName& contract, const Acc
 
 chain_initializer_interface::~chain_initializer_interface() {}
 
+template<typename T>
+void chain_controller::to_binary( const AccountName& scope, const TypeName& type, const fc::variant& value, fc::datastream<T>& ds )const
+{ try {
+   const auto& type_obj = _db.get<type_object,by_scope_name>( boost::make_tuple(scope, type) );
+   const auto& obj = value.get_object();
+
+   if( type_obj.base != TypeName() ) {
+      assert( type_obj.base_scope != AccountName() );
+      to_binary( type_obj.base_scope, type_obj.base, value, ds );
+   }
+
+   for( const auto& field : type_obj.fields ) {
+      // TODO: check to see if field.type is a built in type, otherwise recurse
+#warning TODO: determine whether a stack overflow could be caused by nesting types, may be mitigated by controlling sizeof value
+      to_binary( scope, field.type, obj[field.name], ds );
+   }
+} FC_CAPTURE_AND_RETHROW( (scope)(type)(value) ) }
+
+/**
+ *  This method should look up the type description in the chainstate and then use that description to unpack
+ *  buffer into a fc::variant.  
+ *
+ *  @pre   type is registered in type_index
+ *  @post  to_binary( scope, type, to_variant( scope, type, buffer ) ) == buffer
+ */
+fc::variant chain_controller::to_variant( const AccountName& scope, const TypeName& type, fc::datastream<const char*>& ds )const {
+   fc::mutable_variant_object mvo;
+   to_variant( scope, type, ds, mvo );
+   return fc::variant(std::move( mvo )) ;
+}
+void chain_controller::to_variant( const AccountName& scope, const TypeName& type, fc::datastream<const char*>& ds, fc::mutable_variant_object& mvo )const {
+try {
+   const auto& type_obj = _db.get<type_object,by_scope_name>( boost::make_tuple(scope, type) );
+
+   if( type_obj.base != TypeName() ) {
+      assert( type_obj.base_scope != AccountName() );
+      to_variant( type_obj.base_scope, type_obj.base, ds, mvo );
+   }
+
+   for( const auto& field : type_obj.fields ) {
+      /// TODO: check to see if type is built in
+      mvo.set(field.type, to_variant( scope, field.type, ds ) );
+   }
+} FC_CAPTURE_AND_RETHROW( (scope)(type) ) }
+
+Bytes chain_controller::to_binary( const AccountName& scope, const TypeName& type, const fc::variant& value )const {
+try {
+   fc::datastream<size_t> bytes_size;
+   to_binary( scope, type, value, bytes_size );
+   Bytes temp(bytes_size.tellp());
+   fc::datastream<char*>  ds( temp.data(), temp.size() );
+   to_binary( scope, type, value, ds );
+   return temp;
+} FC_CAPTURE_AND_RETHROW( (scope)(type)(value) ) }
+
+
 } }

libraries/chain/include/eos/chain/chain_controller.hpp

@@ -248,14 +248,23 @@ namespace eos { namespace chain {
 
          uint32_t last_irreversible_block_num() const;
 
-         void debug_dump();
-         void apply_debug_updates();
-         void debug_update(const fc::variant_object& update);
+         void         from_variant( const AccountName& scope, const TypeName& type, 
+                                    const fc::variant& value, 
+                                    fc::datastream<char*>& ds )const;
+         types::Bytes to_binary( const AccountName& scope, const TypeName& type, const fc::variant& value  )const;
+         fc::variant  to_variant( const AccountName& scope, const TypeName& type, fc::datastream<const char*>& ds )const;
 
    protected:
          const chainbase::database& get_database() const { return _db; }
          
    private:
+         template<typename T>
+         void to_binary( const AccountName& scope, const TypeName& type, const fc::variant& value, fc::datastream<T>& ds )const;
+         void to_variant( const AccountName& scope, const TypeName& type, 
+                          fc::datastream<const char*>& ds, 
+                          fc::mutable_variant_object& mvo )const;
+
+
          /// Reset the object graph in-memory
          void initialize_indexes();
          void initialize_chain(chain_initializer_interface& starter);

libraries/chain/include/eos/chain/type_object.hpp

@@ -32,9 +32,9 @@ namespace eos { namespace chain {
       OBJECT_CTOR(type_object, (fields))
 
       id_type               id;
-      AccountName          scope;
+      AccountName           scope;
       TypeName              name;
-      AccountName          base_scope;
+      AccountName           base_scope;
       TypeName              base;
       shared_vector<Field>  fields;
    };

libraries/chain/include/eos/chain/types.hpp

@@ -191,6 +191,7 @@ namespace eos { namespace chain {
    using generated_transaction_id_type = fc::sha256;
    using signature_type = fc::ecc::compact_signature;
    using weight_type = uint16_t;
+   using Bytes = types::Bytes;
 
    using public_key_type = eos::types::PublicKey;