Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 1 年 前同步成功

通知 2299
Star 20931
Fork 5422
P
Paddle
提交 106620ea 编写于 作者: Y Yu Yang
浏览文件
操作

Merge branch 'feature/use_std_cmake' into feature/c_api

上级 3fcd81fd a873a40e
显示空白变更内容
内联 并排
Showing with 468 addition and 885 deletion
CMakeLists.txt
浏览文件 @ 106620ea
......@@ -26,7 +26,7 @@ find_package(NumPy REQUIRED)
find_package(Threads REQUIRED)
find_package(AVX QUIET)
find_package(Glog REQUIRED)
find_package(Gflags COMPONENTS nothreads_static REQUIRED)
find_package(gflags COMPONENTS nothreads_static REQUIRED)
find_package(GTest)
find_package(Sphinx)
find_package(Doxygen)
......
cmake/FindGflags.cmake 已删除 100644 → 0
浏览文件 @ 3fcd81fd
# Ceres Solver - A fast non-linear least squares minimizer
# Copyright 2015 Google Inc. All rights reserved.
# http://ceres-solver.org/
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# * Neither the name of Google Inc. nor the names of its contributors may be
# used to endorse or promote products derived from this software without
# specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
# Author: alexs.mac@gmail.com (Alex Stewart)
#
# FindGflags.cmake - Find Google gflags logging library.
#
# This module will attempt to find gflags, either via an exported CMake
# configuration (generated by gflags >= 2.1 which are built with CMake), or
# by performing a standard search for all gflags components. The order of
# precedence for these two methods of finding gflags is controlled by:
# GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION.
#
# This module defines the following variables:
#
# GFLAGS_FOUND: TRUE iff gflags is found.
# GFLAGS_INCLUDE_DIRS: Include directories for gflags.
# GFLAGS_LIBRARIES: Libraries required to link gflags.
# GFLAGS_NAMESPACE: The namespace in which gflags is defined. In versions of
# gflags < 2.1, this was google, for versions >= 2.1 it is
# by default gflags, although can be configured when building
# gflags to be something else (i.e. google for legacy
# compatibility).
#
# The following variables control the behaviour of this module when an exported
# gflags CMake configuration is not found.
#
# GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION: TRUE/FALSE, iff TRUE then
# then prefer using an exported CMake configuration
# generated by gflags >= 2.1 over searching for the
# gflags components manually. Otherwise (FALSE)
# ignore any exported gflags CMake configurations and
# always perform a manual search for the components.
# Default: TRUE iff user does not define this variable
# before we are called, and does NOT specify either
# GFLAGS_INCLUDE_DIR_HINTS or GFLAGS_LIBRARY_DIR_HINTS
# otherwise FALSE.
# GFLAGS_INCLUDE_DIR_HINTS: List of additional directories in which to
# search for gflags includes, e.g: /timbuktu/include.
# GFLAGS_LIBRARY_DIR_HINTS: List of additional directories in which to
# search for gflags libraries, e.g: /timbuktu/lib.
#
# The following variables are also defined by this module, but in line with
# CMake recommended FindPackage() module style should NOT be referenced directly
# by callers (use the plural variables detailed above instead). These variables
# do however affect the behaviour of the module via FIND_[PATH/LIBRARY]() which
# are NOT re-called (i.e. search for library is not repeated) if these variables
# are set with valid values _in the CMake cache_. This means that if these
# variables are set directly in the cache, either by the user in the CMake GUI,
# or by the user passing -DVAR=VALUE directives to CMake when called (which
# explicitly defines a cache variable), then they will be used verbatim,
# bypassing the HINTS variables and other hard-coded search locations.
#
# GFLAGS_INCLUDE_DIR: Include directory for gflags, not including the
# include directory of any dependencies.
# GFLAGS_LIBRARY: gflags library, not including the libraries of any
# dependencies.
# Reset CALLERS_CMAKE_FIND_LIBRARY_PREFIXES to its value when FindGflags was
# invoked, necessary for MSVC.
macro(GFLAGS_RESET_FIND_LIBRARY_PREFIX)
if (MSVC)
set(CMAKE_FIND_LIBRARY_PREFIXES "${CALLERS_CMAKE_FIND_LIBRARY_PREFIXES}")
endif (MSVC)
endmacro(GFLAGS_RESET_FIND_LIBRARY_PREFIX)
# Called if we failed to find gflags or any of it's required dependencies,
# unsets all public (designed to be used externally) variables and reports
# error message at priority depending upon [REQUIRED/QUIET/<NONE>] argument.
macro(GFLAGS_REPORT_NOT_FOUND REASON_MSG)
unset(GFLAGS_FOUND)
unset(GFLAGS_INCLUDE_DIRS)
unset(GFLAGS_LIBRARIES)
# Do not use unset, as we want to keep GFLAGS_NAMESPACE in the cache,
# but simply clear its value.
set(GFLAGS_NAMESPACE "" CACHE STRING
"gflags namespace (google or gflags)" FORCE)
# Make results of search visible in the CMake GUI if gflags has not
# been found so that user does not have to toggle to advanced view.
mark_as_advanced(CLEAR GFLAGS_INCLUDE_DIR
GFLAGS_LIBRARY
GFLAGS_NAMESPACE)
gflags_reset_find_library_prefix()
# Note <package>_FIND_[REQUIRED/QUIETLY] variables defined by FindPackage()
# use the camelcase library name, not uppercase.
if (Gflags_FIND_QUIETLY)
message(STATUS "Failed to find gflags - " ${REASON_MSG} ${ARGN})
elseif (Gflags_FIND_REQUIRED)
message(FATAL_ERROR "Failed to find gflags - " ${REASON_MSG} ${ARGN})
else()
# Neither QUIETLY nor REQUIRED, use no priority which emits a message
# but continues configuration and allows generation.
message("-- Failed to find gflags - " ${REASON_MSG} ${ARGN})
endif ()
return()
endmacro(GFLAGS_REPORT_NOT_FOUND)
# Verify that all variable names passed as arguments are defined (can be empty
# but must be defined) or raise a fatal error.
macro(GFLAGS_CHECK_VARS_DEFINED)
foreach(CHECK_VAR ${ARGN})
if (NOT DEFINED ${CHECK_VAR})
message(FATAL_ERROR "Ceres Bug: ${CHECK_VAR} is not defined.")
endif()
endforeach()
endmacro(GFLAGS_CHECK_VARS_DEFINED)
# Use check_cxx_source_compiles() to compile trivial test programs to determine
# the gflags namespace. This works on all OSs except Windows. If using Visual
# Studio, it fails because msbuild forces check_cxx_source_compiles() to use
# CMAKE_BUILD_TYPE=Debug for the test project, which usually breaks detection
# because MSVC requires that the test project use the same build type as gflags,
# which would normally be built in Release.
#
# Defines: GFLAGS_NAMESPACE in the caller's scope with the detected namespace,
# which is blank (empty string, will test FALSE is CMake conditionals)
# if detection failed.
function(GFLAGS_CHECK_GFLAGS_NAMESPACE_USING_TRY_COMPILE)
# Verify that all required variables are defined.
gflags_check_vars_defined(
GFLAGS_INCLUDE_DIR GFLAGS_LIBRARY)
# Ensure that GFLAGS_NAMESPACE is always unset on completion unless
# we explicitly set if after having the correct namespace.
set(GFLAGS_NAMESPACE "" PARENT_SCOPE)
include(CheckCXXSourceCompiles)
# Setup include path & link library for gflags for CHECK_CXX_SOURCE_COMPILES.
set(CMAKE_REQUIRED_INCLUDES ${GFLAGS_INCLUDE_DIR})
set(CMAKE_REQUIRED_LIBRARIES ${GFLAGS_LIBRARY} ${GFLAGS_LINK_LIBRARIES})
# First try the (older) google namespace. Note that the output variable
# MUST be unique to the build type as otherwise the test is not repeated as
# it is assumed to have already been performed.
check_cxx_source_compiles(
"#include <gflags/gflags.h>
int main(int argc, char * argv[]) {
google::ParseCommandLineFlags(&argc, &argv, true);
return 0;
}"
GFLAGS_IN_GOOGLE_NAMESPACE)
if (GFLAGS_IN_GOOGLE_NAMESPACE)
set(GFLAGS_NAMESPACE google PARENT_SCOPE)
return()
endif()
# Try (newer) gflags namespace instead. Note that the output variable
# MUST be unique to the build type as otherwise the test is not repeated as
# it is assumed to have already been performed.
set(CMAKE_REQUIRED_INCLUDES ${GFLAGS_INCLUDE_DIR})
set(CMAKE_REQUIRED_LIBRARIES ${GFLAGS_LIBRARY} ${GFLAGS_LINK_LIBRARIES})
check_cxx_source_compiles(
"#include <gflags/gflags.h>
int main(int argc, char * argv[]) {
gflags::ParseCommandLineFlags(&argc, &argv, true);
return 0;
}"
GFLAGS_IN_GFLAGS_NAMESPACE)
if (GFLAGS_IN_GFLAGS_NAMESPACE)
set(GFLAGS_NAMESPACE gflags PARENT_SCOPE)
return()
endif (GFLAGS_IN_GFLAGS_NAMESPACE)
endfunction(GFLAGS_CHECK_GFLAGS_NAMESPACE_USING_TRY_COMPILE)
# Use regex on the gflags headers to attempt to determine the gflags namespace.
# Checks both gflags.h (contained namespace on versions < 2.1.2) and
# gflags_declare.h, which contains the namespace on versions >= 2.1.2.
# In general, this method should only be used when
# GFLAGS_CHECK_GFLAGS_NAMESPACE_USING_TRY_COMPILE() cannot be used, or has
# failed.
#
# Defines: GFLAGS_NAMESPACE in the caller's scope with the detected namespace,
# which is blank (empty string, will test FALSE is CMake conditionals)
# if detection failed.
function(GFLAGS_CHECK_GFLAGS_NAMESPACE_USING_REGEX)
# Verify that all required variables are defined.
gflags_check_vars_defined(GFLAGS_INCLUDE_DIR)
# Ensure that GFLAGS_NAMESPACE is always undefined on completion unless
# we explicitly set if after having the correct namespace.
set(GFLAGS_NAMESPACE "" PARENT_SCOPE)
# Scan gflags.h to identify what namespace gflags was built with. On
# versions of gflags < 2.1.2, gflags.h was configured with the namespace
# directly, on >= 2.1.2, gflags.h uses the GFLAGS_NAMESPACE #define which
# is defined in gflags_declare.h, we try each location in turn.
set(GFLAGS_HEADER_FILE ${GFLAGS_INCLUDE_DIR}/gflags/gflags.h)
if (NOT EXISTS ${GFLAGS_HEADER_FILE})
gflags_report_not_found(
"Could not find file: ${GFLAGS_HEADER_FILE} "
"containing namespace information in gflags install located at: "
"${GFLAGS_INCLUDE_DIR}.")
endif()
file(READ ${GFLAGS_HEADER_FILE} GFLAGS_HEADER_FILE_CONTENTS)
string(REGEX MATCH "namespace [A-Za-z]+"
GFLAGS_NAMESPACE "${GFLAGS_HEADER_FILE_CONTENTS}")
string(REGEX REPLACE "namespace ([A-Za-z]+)" "\\1"
GFLAGS_NAMESPACE "${GFLAGS_NAMESPACE}")
if (NOT GFLAGS_NAMESPACE)
gflags_report_not_found(
"Failed to extract gflags namespace from header file: "
"${GFLAGS_HEADER_FILE}.")
endif (NOT GFLAGS_NAMESPACE)
if (GFLAGS_NAMESPACE STREQUAL "google" OR
GFLAGS_NAMESPACE STREQUAL "gflags")
# Found valid gflags namespace from gflags.h.
set(GFLAGS_NAMESPACE "${GFLAGS_NAMESPACE}" PARENT_SCOPE)
return()
endif()
# Failed to find gflags namespace from gflags.h, gflags is likely a new
# version, check gflags_declare.h, which in newer versions (>= 2.1.2) contains
# the GFLAGS_NAMESPACE #define, which is then referenced in gflags.h.
set(GFLAGS_DECLARE_FILE ${GFLAGS_INCLUDE_DIR}/gflags/gflags_declare.h)
if (NOT EXISTS ${GFLAGS_DECLARE_FILE})
gflags_report_not_found(
"Could not find file: ${GFLAGS_DECLARE_FILE} "
"containing namespace information in gflags install located at: "
"${GFLAGS_INCLUDE_DIR}.")
endif()
file(READ ${GFLAGS_DECLARE_FILE} GFLAGS_DECLARE_FILE_CONTENTS)
string(REGEX MATCH "#define GFLAGS_NAMESPACE [A-Za-z]+"
GFLAGS_NAMESPACE "${GFLAGS_DECLARE_FILE_CONTENTS}")
string(REGEX REPLACE "#define GFLAGS_NAMESPACE ([A-Za-z]+)" "\\1"
GFLAGS_NAMESPACE "${GFLAGS_NAMESPACE}")
if (NOT GFLAGS_NAMESPACE)
gflags_report_not_found(
"Failed to extract gflags namespace from declare file: "
"${GFLAGS_DECLARE_FILE}.")
endif (NOT GFLAGS_NAMESPACE)
if (GFLAGS_NAMESPACE STREQUAL "google" OR
GFLAGS_NAMESPACE STREQUAL "gflags")
# Found valid gflags namespace from gflags.h.
set(GFLAGS_NAMESPACE "${GFLAGS_NAMESPACE}" PARENT_SCOPE)
return()
endif()
endfunction(GFLAGS_CHECK_GFLAGS_NAMESPACE_USING_REGEX)
# -----------------------------------------------------------------
# By default, if the user has expressed no preference for using an exported
# gflags CMake configuration over performing a search for the installed
# components, and has not specified any hints for the search locations, then
# prefer a gflags exported configuration if available.
if (NOT DEFINED GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION
AND NOT GFLAGS_INCLUDE_DIR_HINTS
AND NOT GFLAGS_LIBRARY_DIR_HINTS)
message(STATUS "No preference for use of exported gflags CMake configuration "
"set, and no hints for include/library directories provided. "
"Defaulting to preferring an installed/exported gflags CMake configuration "
"if available.")
set(GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION TRUE)
endif()
if (GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION)
# Try to find an exported CMake configuration for gflags, as generated by
# gflags versions >= 2.1.
#
# We search twice, s/t we can invert the ordering of precedence used by
# find_package() for exported package build directories, and installed
# packages (found via CMAKE_SYSTEM_PREFIX_PATH), listed as items 6) and 7)
# respectively in [1].
#
# By default, exported build directories are (in theory) detected first, and
# this is usually the case on Windows. However, on OS X & Linux, the install
# path (/usr/local) is typically present in the PATH environment variable
# which is checked in item 4) in [1] (i.e. before both of the above, unless
# NO_SYSTEM_ENVIRONMENT_PATH is passed). As such on those OSs installed
# packages are usually detected in preference to exported package build
# directories.
#
# To ensure a more consistent response across all OSs, and as users usually
# want to prefer an installed version of a package over a locally built one
# where both exist (esp. as the exported build directory might be removed
# after installation), we first search with NO_CMAKE_PACKAGE_REGISTRY which
# means any build directories exported by the user are ignored, and thus
# installed directories are preferred. If this fails to find the package
# we then research again, but without NO_CMAKE_PACKAGE_REGISTRY, so any
# exported build directories will now be detected.
#
# To prevent confusion on Windows, we also pass NO_CMAKE_BUILDS_PATH (which
# is item 5) in [1]), to not preferentially use projects that were built
# recently with the CMake GUI to ensure that we always prefer an installed
# version if available.
#
# [1] http://www.cmake.org/cmake/help/v2.8.11/cmake.html#command:find_package
find_package(gflags QUIET
NO_MODULE
NO_CMAKE_PACKAGE_REGISTRY
NO_CMAKE_BUILDS_PATH)
if (gflags_FOUND)
message(STATUS "Found installed version of gflags: ${gflags_DIR}")
else(gflags_FOUND)
# Failed to find an installed version of gflags, repeat search allowing
# exported build directories.
message(STATUS "Failed to find installed gflags CMake configuration, "
"searching for gflags build directories exported with CMake.")
# Again pass NO_CMAKE_BUILDS_PATH, as we know that gflags is exported and
# do not want to treat projects built with the CMake GUI preferentially.
find_package(gflags QUIET
NO_MODULE
NO_CMAKE_BUILDS_PATH)
if (gflags_FOUND)
message(STATUS "Found exported gflags build directory: ${gflags_DIR}")
endif(gflags_FOUND)
endif(gflags_FOUND)
set(FOUND_INSTALLED_GFLAGS_CMAKE_CONFIGURATION ${gflags_FOUND})
# gflags v2.1 - 2.1.2 shipped with a bug in their gflags-config.cmake [1]
# whereby gflags_LIBRARIES = "gflags", but there was no imported target
# called "gflags", they were called: gflags[_nothreads]-[static/shared].
# As this causes linker errors when gflags is not installed in a location
# on the current library paths, detect if this problem is present and
# fix it.
#
# [1] https://github.com/gflags/gflags/issues/110
if (gflags_FOUND)
# NOTE: This is not written as additional conditions in the outer
# if (gflags_FOUND) as the NOT TARGET "${gflags_LIBRARIES}"
# condition causes problems if gflags is not found.
if (${gflags_VERSION} VERSION_LESS 2.1.3 AND
NOT TARGET "${gflags_LIBRARIES}")
message(STATUS "Detected broken gflags install in: ${gflags_DIR}, "
"version: ${gflags_VERSION} <= 2.1.2 which defines gflags_LIBRARIES = "
"${gflags_LIBRARIES} which is not an imported CMake target, see: "
"https://github.com/gflags/gflags/issues/110. Attempting to fix by "
"detecting correct gflags target.")
# Ordering here expresses preference for detection, specifically we do not
# want to use the _nothreads variants if the full library is available.
list(APPEND CHECK_GFLAGS_IMPORTED_TARGET_NAMES
gflags-shared gflags-static
gflags_nothreads-shared gflags_nothreads-static)
foreach(CHECK_GFLAGS_TARGET ${CHECK_GFLAGS_IMPORTED_TARGET_NAMES})
if (TARGET ${CHECK_GFLAGS_TARGET})
message(STATUS "Found valid gflags target: ${CHECK_GFLAGS_TARGET}, "
"updating gflags_LIBRARIES.")
set(gflags_LIBRARIES ${CHECK_GFLAGS_TARGET})
break()
endif()
endforeach()
if (NOT TARGET ${gflags_LIBRARIES})
message(STATUS "Failed to fix detected broken gflags install in: "
"${gflags_DIR}, version: ${gflags_VERSION} <= 2.1.2, none of the "
"imported targets for gflags: ${CHECK_GFLAGS_IMPORTED_TARGET_NAMES} "
"are defined. Will continue with a manual search for gflags "
"components. We recommend you build/install a version of gflags > "
"2.1.2 (or master).")
set(FOUND_INSTALLED_GFLAGS_CMAKE_CONFIGURATION FALSE)
endif()
endif()
endif()
if (FOUND_INSTALLED_GFLAGS_CMAKE_CONFIGURATION)
message(STATUS "Detected gflags version: ${gflags_VERSION}")
set(GFLAGS_FOUND ${gflags_FOUND})
set(GFLAGS_INCLUDE_DIR ${gflags_INCLUDE_DIR})
set(GFLAGS_LIBRARY ${gflags_LIBRARIES})
# gflags does not export the namespace in their CMake configuration, so
# use our function to determine what it should be, as it can be either
# gflags or google dependent upon version & configuration.
#
# NOTE: We use the regex method to determine the namespace here, as
# check_cxx_source_compiles() will not use imported targets, which
# is what gflags will be in this case.
gflags_check_gflags_namespace_using_regex()
if (NOT GFLAGS_NAMESPACE)
gflags_report_not_found(
"Failed to determine gflags namespace using regex for gflags "
"version: ${gflags_VERSION} exported here: ${gflags_DIR} using CMake.")
endif (NOT GFLAGS_NAMESPACE)
else (FOUND_INSTALLED_GFLAGS_CMAKE_CONFIGURATION)
message(STATUS "Failed to find an installed/exported CMake configuration "
"for gflags, will perform search for installed gflags components.")
endif (FOUND_INSTALLED_GFLAGS_CMAKE_CONFIGURATION)
endif(GFLAGS_PREFER_EXPORTED_GFLAGS_CMAKE_CONFIGURATION)
if (NOT GFLAGS_FOUND)
# Either failed to find an exported gflags CMake configuration, or user
# told us not to use one. Perform a manual search for all gflags components.
# Handle possible presence of lib prefix for libraries on MSVC, see
# also GFLAGS_RESET_FIND_LIBRARY_PREFIX().
if (MSVC)
# Preserve the caller's original values for CMAKE_FIND_LIBRARY_PREFIXES
# s/t we can set it back before returning.
set(CALLERS_CMAKE_FIND_LIBRARY_PREFIXES "${CMAKE_FIND_LIBRARY_PREFIXES}")
# The empty string in this list is important, it represents the case when
# the libraries have no prefix (shared libraries / DLLs).
set(CMAKE_FIND_LIBRARY_PREFIXES "lib" "" "${CMAKE_FIND_LIBRARY_PREFIXES}")
endif (MSVC)
# Search user-installed locations first, so that we prefer user installs
# to system installs where both exist.
list(APPEND GFLAGS_CHECK_INCLUDE_DIRS
/usr/local/include
/usr/local/homebrew/include # Mac OS X
/opt/local/var/macports/software # Mac OS X.
/opt/local/include
/usr/include)
list(APPEND GFLAGS_CHECK_PATH_SUFFIXES
gflags/include # Windows (for C:/Program Files prefix).
gflags/Include ) # Windows (for C:/Program Files prefix).
list(APPEND GFLAGS_CHECK_LIBRARY_DIRS
/usr/local/lib
/usr/local/homebrew/lib # Mac OS X.
/opt/local/lib
/usr/lib)
list(APPEND GFLAGS_CHECK_LIBRARY_SUFFIXES
gflags/lib # Windows (for C:/Program Files prefix).
gflags/Lib ) # Windows (for C:/Program Files prefix).
# Search supplied hint directories first if supplied.
find_path(GFLAGS_INCLUDE_DIR
NAMES gflags/gflags.h
PATHS ${GFLAGS_INCLUDE_DIR_HINTS}
${GFLAGS_CHECK_INCLUDE_DIRS}
PATH_SUFFIXES ${GFLAGS_CHECK_PATH_SUFFIXES})
if (NOT GFLAGS_INCLUDE_DIR OR
NOT EXISTS ${GFLAGS_INCLUDE_DIR})
gflags_report_not_found(
"Could not find gflags include directory, set GFLAGS_INCLUDE_DIR "
"to directory containing gflags/gflags.h")
endif (NOT GFLAGS_INCLUDE_DIR OR
NOT EXISTS ${GFLAGS_INCLUDE_DIR})
find_library(GFLAGS_LIBRARY NAMES gflags
PATHS ${GFLAGS_LIBRARY_DIR_HINTS}
${GFLAGS_CHECK_LIBRARY_DIRS}
PATH_SUFFIXES ${GFLAGS_CHECK_LIBRARY_SUFFIXES})
if (NOT GFLAGS_LIBRARY OR
NOT EXISTS ${GFLAGS_LIBRARY})
gflags_report_not_found(
"Could not find gflags library, set GFLAGS_LIBRARY "
"to full path to libgflags.")
endif (NOT GFLAGS_LIBRARY OR
NOT EXISTS ${GFLAGS_LIBRARY})
# gflags typically requires a threading library (which is OS dependent), note
# that this defines the CMAKE_THREAD_LIBS_INIT variable. If we are able to
# detect threads, we assume that gflags requires it.
find_package(Threads QUIET)
set(GFLAGS_LINK_LIBRARIES ${CMAKE_THREAD_LIBS_INIT})
# On Windows (including MinGW), the Shlwapi library is used by gflags if
# available.
if (WIN32)
include(CheckIncludeFileCXX)
check_include_file_cxx("shlwapi.h" HAVE_SHLWAPI)
if (HAVE_SHLWAPI)
list(APPEND GFLAGS_LINK_LIBRARIES shlwapi.lib)
endif(HAVE_SHLWAPI)
endif (WIN32)
# Mark internally as found, then verify. GFLAGS_REPORT_NOT_FOUND() unsets
# if called.
set(GFLAGS_FOUND TRUE)
# Identify what namespace gflags was built with.
if (GFLAGS_INCLUDE_DIR AND NOT GFLAGS_NAMESPACE)
# To handle Windows peculiarities / CMake bugs on MSVC we try two approaches
# to detect the gflags namespace:
#
# 1) Try to use check_cxx_source_compiles() to compile a trivial program
# with the two choices for the gflags namespace.
#
# 2) [In the event 1) fails] Use regex on the gflags headers to try to
# determine the gflags namespace. Whilst this is less robust than 1),
# it does avoid any interaction with msbuild.
gflags_check_gflags_namespace_using_try_compile()
if (NOT GFLAGS_NAMESPACE)
# Failed to determine gflags namespace using check_cxx_source_compiles()
# method, try and obtain it using regex on the gflags headers instead.
message(STATUS "Failed to find gflags namespace using using "
"check_cxx_source_compiles(), trying namespace regex instead, "
"this is expected on Windows.")
gflags_check_gflags_namespace_using_regex()
if (NOT GFLAGS_NAMESPACE)
gflags_report_not_found(
"Failed to determine gflags namespace either by "
"check_cxx_source_compiles(), or namespace regex.")
endif (NOT GFLAGS_NAMESPACE)
endif (NOT GFLAGS_NAMESPACE)
endif (GFLAGS_INCLUDE_DIR AND NOT GFLAGS_NAMESPACE)
# Make the GFLAGS_NAMESPACE a cache variable s/t the user can view it, and could
# overwrite it in the CMake GUI.
set(GFLAGS_NAMESPACE "${GFLAGS_NAMESPACE}" CACHE STRING
"gflags namespace (google or gflags)" FORCE)
# gflags does not seem to provide any record of the version in its
# source tree, thus cannot extract version.
# Catch case when caller has set GFLAGS_NAMESPACE in the cache / GUI
# with an invalid value.
if (GFLAGS_NAMESPACE AND
NOT GFLAGS_NAMESPACE STREQUAL "google" AND
NOT GFLAGS_NAMESPACE STREQUAL "gflags")
gflags_report_not_found(
"Caller defined GFLAGS_NAMESPACE:"
" ${GFLAGS_NAMESPACE} is not valid, not google or gflags.")
endif ()
# Catch case when caller has set GFLAGS_INCLUDE_DIR in the cache / GUI and
# thus FIND_[PATH/LIBRARY] are not called, but specified locations are
# invalid, otherwise we would report the library as found.
if (GFLAGS_INCLUDE_DIR AND
NOT EXISTS ${GFLAGS_INCLUDE_DIR}/gflags/gflags.h)
gflags_report_not_found(
"Caller defined GFLAGS_INCLUDE_DIR:"
" ${GFLAGS_INCLUDE_DIR} does not contain gflags/gflags.h header.")
endif (GFLAGS_INCLUDE_DIR AND
NOT EXISTS ${GFLAGS_INCLUDE_DIR}/gflags/gflags.h)
# TODO: This regex for gflags library is pretty primitive, we use lowercase
# for comparison to handle Windows using CamelCase library names, could
# this check be better?
string(TOLOWER "${GFLAGS_LIBRARY}" LOWERCASE_GFLAGS_LIBRARY)
if (GFLAGS_LIBRARY AND
NOT "${LOWERCASE_GFLAGS_LIBRARY}" MATCHES ".*gflags[^/]*")
gflags_report_not_found(
"Caller defined GFLAGS_LIBRARY: "
"${GFLAGS_LIBRARY} does not match gflags.")
endif (GFLAGS_LIBRARY AND
NOT "${LOWERCASE_GFLAGS_LIBRARY}" MATCHES ".*gflags[^/]*")
gflags_reset_find_library_prefix()
endif(NOT GFLAGS_FOUND)
# Set standard CMake FindPackage variables if found.
if (GFLAGS_FOUND)
set(GFLAGS_INCLUDE_DIRS ${GFLAGS_INCLUDE_DIR})
set(GFLAGS_LIBRARIES ${GFLAGS_LIBRARY} ${GFLAGS_LINK_LIBRARIES})
endif (GFLAGS_FOUND)
# Handle REQUIRED / QUIET optional arguments.
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Gflags DEFAULT_MSG
GFLAGS_INCLUDE_DIRS GFLAGS_LIBRARIES GFLAGS_NAMESPACE)
# Only mark internal variables as advanced if we found gflags, otherwise
# leave them visible in the standard GUI for the user to set manually.
if (GFLAGS_FOUND)
mark_as_advanced(FORCE GFLAGS_INCLUDE_DIR
GFLAGS_LIBRARY
GFLAGS_NAMESPACE
gflags_DIR) # Autogenerated by find_package(gflags)
endif (GFLAGS_FOUND)
cmake/util.cmake
浏览文件 @ 106620ea
......@@ -112,6 +112,7 @@ function(link_paddle_exe TARGET_NAME)
${LIBGLOG_LIBRARY}
gflags
${CMAKE_THREAD_LIBS_INIT}
${CMAKE_THREAD_LIBS_INIT}
${CBLAS_LIBS}
${ZLIB_LIBRARIES}
${INTERAL_LIBS}
......
doc/howto/deep_model/rnn/index_cn.rst
浏览文件 @ 106620ea
......@@ -4,6 +4,7 @@ RNN相关模型
.. toctree::
:maxdepth: 1
rnn_config_cn.rst
recurrent_group_cn.md
hierarchical_layer_cn.rst
hrnn_rnn_api_compare_cn.rst
doc/howto/deep_model/rnn/rnn_cn.md 已删除 100644 → 0
浏览文件 @ 3fcd81fd
RNN 配置
=================
本教程将指导你如何在 PaddlePaddle 中配置循环神经网络（RNN）。PaddlePaddle 高度支持灵活和高效的循环神经网络配置。 在本教程中，您将了解如何：
- 准备用来学习循环神经网络的序列数据。
- 配置循环神经网络架构。
- 使用学习完成的循环神经网络模型生成序列。
我们将使用 vanilla 循环神经网络和 sequence to sequence 模型来指导你完成这些步骤。sequence to sequence 模型的代码可以在`demo / seqToseq`找到。
准备序列数据
---------------------
PaddlePaddle 不需要对序列数据进行任何预处理，例如填充。唯一需要做的是将相应类型设置为输入。例如，以下代码段定义了三个输入。 它们都是序列，它们的大小是`src_dict`，`trg_dict`和`trg_dict`：
``` sourceCode
settings.input_types = [
integer_value_sequence(len(settings.src_dict)),
integer_value_sequence(len(settings.trg_dict)),
integer_value_sequence(len(settings.trg_dict))]
```
在`process`函数中，每个`yield`函数将返回三个整数列表。每个整数列表被视为一个整数序列：
``` sourceCode
yield src_ids, trg_ids, trg_ids_next
```
有关如何编写数据提供程序的更多细节描述，请参考 [PyDataProvider2](../../ui/data_provider/index.html)。完整的数据提供文件在 `demo/seqToseq/dataprovider.py`。
配置循环神经网络架构
-----------------------------------------------
### 简单门控循环神经网络(Gated Recurrent Neural Network)
循环神经网络在每个时间步骤顺序地处理序列。下面列出了 LSTM 的架构的示例。
![image](../../../tutorials/sentiment_analysis/bi_lstm.jpg)
一般来说，循环网络从 *t* = 1 到 *t* = *T* 或者反向地从 *t* = *T* 到 *t* = 1 执行以下操作。
*x*<sub>*t* + 1</sub> = *f*<sub>*x*</sub>(*x*<sub>*t*</sub>),*y*<sub>*t*</sub> = *f*<sub>*y*</sub>(*x*<sub>*t*</sub>)
其中 *f*<sub>*x*</sub>(.) 称为**单步函数**（即单时间步执行的函数，step function），而 *f*<sub>*y*</sub>(.) 称为**输出函数**。在 vanilla 循环神经网络中，单步函数和输出函数都非常简单。然而，PaddlePaddle 可以通过修改这两个函数来实现复杂的网络配置。我们将使用 sequence to sequence 模型演示如何配置复杂的循环神经网络模型。在本节中，我们将使用简单的 vanilla 循环神经网络作为使用`recurrent_group`配置简单循环神经网络的例子。 注意，如果你只需要使用简单的RNN，GRU或LSTM，那么推荐使用`grumemory`和`lstmemory`，因为它们的计算效率比`recurrent_group`更高。
对于 vanilla RNN，在每个时间步长，**单步函数**为：
*x*<sub>*t* + 1</sub> = *W*<sub>*x*</sub>*x*<sub>*t*</sub> + *W*<sub>*i*</sub>*I*<sub>*t*</sub> + *b*
其中 *x*<sub>*t*</sub> 是RNN状态，并且 *I*<sub>*t*</sub> 是输入，*W*<sub>*x*</sub> 和 *W*<sub>*i*</sub> 分别是RNN状态和输入的变换矩阵。*b* 是偏差。它的**输出函数**只需要*x*<sub>*t*</sub>作为输出。
`recurrent_group`是构建循环神经网络的最重要的工具。 它定义了**单步函数**，**输出函数**和循环神经网络的输入。注意，这个函数的`step`参数需要实现`step function`（单步函数）和`output function`（输出函数）：
``` sourceCode
def simple_rnn(input,
size=None,
name=None,
reverse=False,
rnn_bias_attr=None,
act=None,
rnn_layer_attr=None):
def __rnn_step__(ipt):
out_mem = memory(name=name, size=size)
rnn_out = mixed_layer(input = [full_matrix_projection(ipt),
full_matrix_projection(out_mem)],
name = name,
bias_attr = rnn_bias_attr,
act = act,
layer_attr = rnn_layer_attr,
size = size)
return rnn_out
return recurrent_group(name='%s_recurrent_group' % name,
step=__rnn_step__,
reverse=reverse,
input=input)
```
PaddlePaddle 使用“Memory”（记忆模块）实现单步函数。**Memory**是在PaddlePaddle中构造循环神经网络时最重要的概念。 Memory是在单步函数中循环使用的状态，例如*x*<sub>*t* + 1</sub> = *f*<sub>*x*</sub>(*x*<sub>*t*</sub>)。 一个Memory包含**输出**和**输入**。当前时间步处的Memory的输出作为下一时间步Memory的输入。Memory也可以具有**boot layer(引导层)**，其输出被用作Memory的初始值。 在我们的例子中，门控循环单元的输出被用作输出Memory。请注意，`rnn_out`层的名称与`out_mem`的名称相同。这意味着`rnn_out` (*x*<sub>*t* + 1</sub>)的输出被用作`out_mem`Memory的**输出**。
Memory也可以是序列。在这种情况下，在每个时间步中，我们有一个序列作为循环神经网络的状态。这在构造非常复杂的循环神经网络时是有用的。 其他高级功能包括定义多个Memory，以及使用子序列来定义分级循环神经网络架构。
我们在函数的结尾返回`rnn_out`。 这意味着 `rnn_out` 层的输出被用作门控循环神经网络的**输出**函数。
### Sequence to Sequence Model with Attention
我们将使用 sequence to sequence model with attention 作为例子演示如何配置复杂的循环神经网络模型。该模型的说明如下图所示。
![image](../../../tutorials/text_generation/encoder-decoder-attention-model.png)
在这个模型中，源序列 *S* = {*s*<sub>1</sub>, …, *s*<sub>*T*</sub>} 用双向门控循环神经网络编码。双向门控循环神经网络的隐藏状态 *H*<sub>*S*</sub> = {*H*<sub>1</sub>, …, *H*<sub>*T*</sub>} 被称为 *编码向量*。解码器是门控循环神经网络。当解读每一个*y*<sub>*t*</sub>时, 这个门控循环神经网络生成一系列权重 *W*<sub>*S*</sub><sup>*t*</sup> = {*W*<sub>1</sub><sup>*t*</sup>, …, *W*<sub>*T*</sub><sup>*t*</sup>}, 用于计算编码向量的加权和。加权和用来生成*y*<sub>*t*</sub>。
模型的编码器部分如下所示。它叫做`grumemory`来表示门控循环神经网络。如果网络架构简单，那么推荐使用循环神经网络的方法，因为它比 `recurrent_group` 更快。我们已经实现了大多数常用的循环神经网络架构，可以参考 [Layers](../../ui/api/trainer_config_helpers/layers_index.html) 了解更多细节。
我们还将编码向量投射到 `decoder_size` 维空间。这通过获得反向循环网络的第一个实例，并将其投射到 `decoder_size` 维空间完成：
``` sourceCode
# 定义源语句的数据层
src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
# 计算每个词的词向量
src_embedding = embedding_layer(
input=src_word_id,
size=word_vector_dim,
param_attr=ParamAttr(name='_source_language_embedding'))
# 应用前向循环神经网络
src_forward = grumemory(input=src_embedding, size=encoder_size)
# 应用反向递归神经网络（reverse=True表示反向循环神经网络）
src_backward = grumemory(input=src_embedding,
size=encoder_size,
reverse=True)
# 将循环神经网络的前向和反向部分混合在一起
encoded_vector = concat_layer(input=[src_forward, src_backward])
# 投射编码向量到 decoder_size
encoder_proj = mixed_layer(input = [full_matrix_projection(encoded_vector)],
size = decoder_size)
# 计算反向RNN的第一个实例
backward_first = first_seq(input=src_backward)
# 投射反向RNN的第一个实例到 decoder size
decoder_boot = mixed_layer(input=[full_matrix_projection(backward_first)], size=decoder_size, act=TanhActivation())
```
解码器使用 `recurrent_group` 来定义循环神经网络。单步函数和输出函数在 `gru_decoder_with_attention` 中定义：
``` sourceCode
group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
StaticInput(input=encoded_proj,is_seq=True)]
trg_embedding = embedding_layer(
input=data_layer(name='target_language_word',
size=target_dict_dim),
size=word_vector_dim,
param_attr=ParamAttr(name='_target_language_embedding'))
group_inputs.append(trg_embedding)
# 对于配备有注意力机制的解码器，在训练中，
# 目标向量（groudtruth）是数据输入，
# 而源序列的编码向量可以被无边界的memory访问
# StaticInput 意味着不同时间步的输入都是相同的值，
# 否则它以一个序列输入，不同时间步的输入是不同的。
# 所有输入序列应该有相同的长度。
decoder = recurrent_group(name=decoder_group_name,
step=gru_decoder_with_attention,
input=group_inputs)
```
单步函数的实现如下所示。首先，它定义解码网络的**Memory**。然后定义 attention，门控循环单元单步函数和输出函数：
``` sourceCode
def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
# 定义解码器的Memory
# Memory的输出定义在 gru_step 内
# 注意 gru_step 应该与它的Memory名字相同
decoder_mem = memory(name='gru_decoder',
size=decoder_size,
boot_layer=decoder_boot)
# 计算 attention 加权编码向量
context = simple_attention(encoded_sequence=enc_vec,
encoded_proj=enc_proj,
decoder_state=decoder_mem)
# 混合当前词向量和attention加权编码向量
decoder_inputs = mixed_layer(inputs = [full_matrix_projection(context),
full_matrix_projection(current_word)],
size = decoder_size * 3)
# 定义门控循环单元循环神经网络单步函数
gru_step = gru_step_layer(name='gru_decoder',
input=decoder_inputs,
output_mem=decoder_mem,
size=decoder_size)
# 定义输出函数
out = mixed_layer(input=[full_matrix_projection(input=gru_step)],
size=target_dict_dim,
bias_attr=True,
act=SoftmaxActivation())
return out
```
生成序列
-----------------
训练模型后，我们可以使用它来生成序列。通常的做法是使用**beam search** 生成序列。以下代码片段定义 beam search 算法。注意，`beam_search` 函数假设 `step` 的输出函数返回的是下一个时刻输出词的 softmax 归一化概率向量。我们对模型进行了以下更改。
- 使用 `GeneratedInput` 来表示 trg\_embedding。 `GeneratedInput` 将上一时间步所生成的词的向量来作为当前时间步的输入。
- 使用 `beam_search` 函数。这个函数需要设置：
- `bos_id`: 开始标记。每个句子都以开始标记开头。
- `eos_id`: 结束标记。每个句子都以结束标记结尾。
- `beam_size`: beam search 算法中的beam大小。
- `max_length`: 生成序列的最大长度。
- 使用 `seqtext_printer_evaluator` 根据索引矩阵和字典打印文本。这个函数需要设置：
- `id_input`: 数据的整数ID，用于标识生成的文件中的相应输出。
- `dict_file`: 用于将词ID转换为词的字典文件。
- `result_file`: 生成结果文件的路径。
代码如下：
``` sourceCode
group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
StaticInput(input=encoded_proj,is_seq=True)]
# 在生成时，解码器基于编码源序列和最后生成的目标词预测下一目标词。
# 编码源序列（编码器输出）必须由只读Memory的 StaticInput 指定。
# 这里， GeneratedInputs 自动获取上一个生成的词，并在最开始初始化为起始词，如 <s>。
trg_embedding = GeneratedInput(
size=target_dict_dim,
embedding_name='_target_language_embedding',
embedding_size=word_vector_dim)
group_inputs.append(trg_embedding)
beam_gen = beam_search(name=decoder_group_name,
step=gru_decoder_with_attention,
input=group_inputs,
bos_id=0, # Beginnning token.
eos_id=1, # End of sentence token.
beam_size=beam_size,
max_length=max_length)
seqtext_printer_evaluator(input=beam_gen,
id_input=data_layer(name="sent_id", size=1),
dict_file=trg_dict_path,
result_file=gen_trans_file)
outputs(beam_gen)
```
注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 [Semantic Role Labeling Demo](../../demo/semantic_role_labeling/index.html) 了解更多详细信息。
完整的配置文件在`demo/seqToseq/seqToseq_net.py`。
doc/howto/deep_model/rnn_config_cn.rst → doc/howto/deep_model/rnn/rnn_config_cn.rst
浏览文件 @ 106620ea
RNN 配置
RNN配置
========
本教程将指导你如何在 PaddlePaddle
......@@ -20,7 +20,7 @@ PaddlePaddle
不需要对序列数据进行任何预处理，例如填充。唯一需要做的是将相应类型设置为输入。例如，以下代码段定义了三个输入。
它们都是序列，它们的大小是\ ``src_dict``\ ，\ ``trg_dict``\ 和\ ``trg_dict``\ ：
.. code:: sourcecode
.. code:: python
settings.input_types = [
integer_value_sequence(len(settings.src_dict)),
......@@ -29,12 +29,11 @@ PaddlePaddle
在\ ``process``\ 函数中，每个\ ``yield``\ 函数将返回三个整数列表。每个整数列表被视为一个整数序列：
.. code:: sourcecode
.. code:: python
yield src_ids, trg_ids, trg_ids_next
有关如何编写数据提供程序的更多细节描述，请参考
`PyDataProvider2 <../../ui/data_provider/index.html>`__\ 。完整的数据提供文件在
有关如何编写数据提供程序的更多细节描述，请参考 :ref:`api_pydataprovider2` 。完整的数据提供文件在
``demo/seqToseq/dataprovider.py``\ 。
配置循环神经网络架构
......@@ -45,18 +44,17 @@ PaddlePaddle
循环神经网络在每个时间步骤顺序地处理序列。下面列出了 LSTM 的架构的示例。
.. figure:: ../../../tutorials/sentiment_analysis/bi_lstm.jpg
:alt: image
.. image:: ../../../tutorials/sentiment_analysis/bi_lstm.jpg
:align: center
image
一般来说，循环网络从 :math:`t=1` 到 :math:`t=T` 或者反向地从 :math:`t=T` 到 :math:`t=1` 执行以下操作。
一般来说，循环网络从 *t* = 1 到 *t* = *T* 或者反向地从 *t* = *T* 到 *t*
= 1 执行以下操作。
.. math::
*x*\ \ *t* + 1 = *f*\ \ *x*\ (*x*\ \ *t*\ ),\ *y*\ \ *t*\  = *f*\ \ *y*\ (*x*\ \ *t*\ )
x_{t+1} = f_x(x_t), y_t = f_y(x_t)
其中 *f*\ \ *x*\ (.) 称为\ **单步函数**\ （即单时间步执行的函数，step
function），而 *f*\ \ *y*\ (.) 称为\ **输出函数**\ 。在 vanilla
其中 :math:`f_x(.)` 称为\ **单步函数**\ （即单时间步执行的函数，step
function），而 :math:`f_y(.)` 称为\ **输出函数**\ 。在 vanilla
循环神经网络中，单步函数和输出函数都非常简单。然而，PaddlePaddle
可以通过修改这两个函数来实现复杂的网络配置。我们将使用 sequence to
sequence
......@@ -67,16 +65,17 @@ vanilla
对于 vanilla RNN，在每个时间步长，\ **单步函数**\ 为：
*x*\ \ *t* + 1 = *W*\ \ *x*\ \ *x*\ \ *t*\  + *W*\ \ *i*\ \ *I*\ \ *t*\  + *b*
.. math::
其中 *x*\ \ *t*\ 是RNN状态，并且 *I*\ \ *t*\ 是输入，\ *W*\ \ *x*\ 和
*W*\ \ *i*\ 分别是RNN状态和输入的变换矩阵。\ *b*
是偏差。它的\ **输出函数**\ 只需要\ *x*\ \ *t*\ 作为输出。
x_{t+1} = W_x x_t + W_i I_t + b
其中 :math:`x_t` 是RNN状态，并且 :math:`I_t` 是输入，:math:`W_x` 和
:math:`W_i` 分别是RNN状态和输入的变换矩阵。:math:`b` 是偏差。它的\ **输出函数**\ 只需要 :math:`x_t` 作为输出。
``recurrent_group``\ 是构建循环神经网络的最重要的工具。
它定义了\ **单步函数**\ ，\ **输出函数**\ 和循环神经网络的输入。注意，这个函数的\ ``step``\ 参数需要实现\ ``step function``\ （单步函数）和\ ``output function``\ （输出函数）：
.. code:: sourcecode
.. code:: python
def simple_rnn(input,
size=None,
......@@ -102,7 +101,7 @@ vanilla
PaddlePaddle
使用“Memory”（记忆模块）实现单步函数。\ **Memory**\ 是在PaddlePaddle中构造循环神经网络时最重要的概念。
Memory是在单步函数中循环使用的状态，例如\ *x*\ \ *t* + 1 = *f*\ \ *x*\ (*x*\ \ *t*\ )。
Memory是在单步函数中循环使用的状态，例如 :math:`x_{t+1} = f_x(x_t)` 。
一个Memory包含\ **输出**\ 和\ **输入**\ 。当前时间步处的Memory的输出作为下一时间步Memory的输入。Memory也可以具有\ **boot
layer(引导层)**\ ，其输出被用作Memory的初始值。
在我们的例子中，门控循环单元的输出被用作输出Memory。请注意，\ ``rnn_out``\ 层的名称与\ ``out_mem``\ 的名称相同。这意味着\ ``rnn_out``
......@@ -120,30 +119,25 @@ Sequence to Sequence Model with Attention
我们将使用 sequence to sequence model with attention
作为例子演示如何配置复杂的循环神经网络模型。该模型的说明如下图所示。
.. figure:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png
:alt: image
image
.. image:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png
:align: center
在这个模型中，源序列 *S* = {*s*\ 1, …, \ *s*\ \ *T*\ }
在这个模型中，源序列 :math:`S = \{s_1, \dots, s_T\}`
用双向门控循环神经网络编码。双向门控循环神经网络的隐藏状态
*H*\ \ *S*\  = {*H*\ 1, …, \ *H*\ \ *T*\ } 被称为
*编码向量*\ 。解码器是门控循环神经网络。当解读每一个\ *y*\ \ *t*\ 时,
这个门控循环神经网络生成一系列权重
*W*\ \ *S*\ \ *t*\  = {*W*\ 1\ *t*\ , …, \ *W*\ \ *T*\ \ *t*\ },
用于计算编码向量的加权和。加权和用来生成\ *y*\ \ *t*\ 。
:math:`H_S = \{H_1, \dots, H_T\}` 被称为
*编码向量*\ 。解码器是门控循环神经网络。当解读每一个 :math:`y_t` 时,
这个门控循环神经网络生成一系列权重 :math:`W_S^t = \{W_1^t, \dots, W_T^t\}` ,
用于计算编码向量的加权和。加权和用来生成 :math:`y_t` 。
模型的编码器部分如下所示。它叫做\ ``grumemory``\ 来表示门控循环神经网络。如果网络架构简单，那么推荐使用循环神经网络的方法，因为它比
``recurrent_group``
更快。我们已经实现了大多数常用的循环神经网络架构，可以参考
`Layers <../../ui/api/trainer_config_helpers/layers_index.html>`__
了解更多细节。
更快。我们已经实现了大多数常用的循环神经网络架构，可以参考 :ref:`api_trainer_config_helpers_layers` 了解更多细节。
我们还将编码向量投射到 ``decoder_size``
维空间。这通过获得反向循环网络的第一个实例，并将其投射到
``decoder_size`` 维空间完成：
.. code:: sourcecode
.. code:: python
# 定义源语句的数据层
src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
......@@ -174,7 +168,7 @@ Sequence to Sequence Model with Attention
解码器使用 ``recurrent_group`` 来定义循环神经网络。单步函数和输出函数在
``gru_decoder_with_attention`` 中定义：
.. code:: sourcecode
.. code:: python
group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
StaticInput(input=encoded_proj,is_seq=True)]
......@@ -198,7 +192,7 @@ Sequence to Sequence Model with Attention
单步函数的实现如下所示。首先，它定义解码网络的\ **Memory**\ 。然后定义
attention，门控循环单元单步函数和输出函数：
.. code:: sourcecode
.. code:: python
def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
# 定义解码器的Memory
......@@ -253,7 +247,7 @@ attention，门控循环单元单步函数和输出函数：
代码如下：
.. code:: sourcecode
.. code:: python
group_inputs=[StaticInput(input=encoded_vector,is_seq=True),
StaticInput(input=encoded_proj,is_seq=True)]
......@@ -279,9 +273,6 @@ attention，门控循环单元单步函数和输出函数：
result_file=gen_trans_file)
outputs(beam_gen)
注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅
`Semantic Role Labeling
Demo <../../demo/semantic_role_labeling/index.html>`__
了解更多详细信息。
注意，这种生成技术只用于类似解码器的生成过程。如果你正在处理序列标记任务，请参阅 :ref:`semantic_role_labeling` 了解更多详细信息。
完整的配置文件在\ ``demo/seqToseq/seqToseq_net.py``\ 。
doc/howto/dev/new_layer_cn.rst 0 → 100644
浏览文件 @ 106620ea
================
实现新的网络层
================
这份教程展示了如何在PaddlePaddle中实现一个自定义的网络层。在这里我们使用全连接层作为例子来展示实现新网络层所需要的四个步骤。
1. 推导该层前向和后向传递的方程。
2. 实现该层的C++类。
3. 增加梯度检测的单元测试，以保证梯度的正确计算。
4. 封装该层的Python接口。
推导方程
================
首先我们需要推导该网络层的*前向传播*和*后向传播*的方程。前向传播给定输入，计算输出。后向传播给定输出的梯度，计算输入和参数的梯度。
下图是一个全连接层的示意图。在全连接层中，每个输出节点都连接到所有的输入节点上。
.. image:: FullyConnected.jpg
:align: center
:scale: 60 %
一个网络层的前向传播部分把输入转化为相应的输出。
全连接层以一个维度为 :math:`D_i` 的稠密向量作为输入，使用一个尺度为 :math:`D_i \times D_o` 的变换矩阵 :math:`W` 把 :math:`x` 映射到一个维度为 :math:`D_o` 的向量，并在乘积结果上再加上维度为 :math:`D_o` 的偏置向量 :math:`b` 。
.. math::
y = f(W^T x + b)
其中 :math:`f(.)` 是一个非线性的*激活方程*，例如sigmoid， tanh，以及Relu。
变换矩阵 :math:`W` 和偏置向量 :math:`b` 是该网络层的*参数*。一个网络层的参数是在*反向传播*时被训练的。反向传播根据输出的梯度，分别计算每个参数的梯度，以及输入的梯度。优化器则用链式法则来对每个参数计算损失函数的梯度。
假设损失函数是 :math:`c(y)` ，那么
.. math::
\frac{\partial c(y)}{\partial x} = \frac{\partial c(y)}{\partial y} \frac{\partial y}{\partial x}
假设 :math:`z = f(W^T x + b)` ，那么
.. math::
\frac{\partial y}{\partial z} = \frac{\partial f(z)}{\partial z}
PaddlePaddle的base layer类可以自动计算上面的导数。
因此，对全连接层来说，我们需要计算：
.. math::
\frac{\partial z}{\partial x} = W, \frac{\partial z_j}{\partial W_{ij}} = x_i, \frac{\partial z}{\partial b} = \mathbf 1
其中 :math:`\mathbf 1` 是一个全1的向量， :math:`W_{ij}` 是矩阵 :math:`W` 第i行第j列的数值， :math:`z_j` 是向量 :math:`z` 的第j个值， :math:`x_i` 是向量 :math:`x` 的第i个值。
最后我们使用链式法则计算 :math:`\frac{\partial z}{\partial x}` 以及 :math:`\frac{\partial z}{\partial W}` 。计算的细节将在下面的小节给出。
实现C++类
===================
一个网络层的C++类需要实现初始化，前向和后向。全连接层的实现位于:code:`paddle/gserver/layers/FullyConnectedLayer.h`及:code:`paddle/gserver/layers/FullyConnectedLayer.cpp`。这里我们展示一份简化过的代码。
这个类需要继承 :code:`paddle::Layer` 这个基类，并且需要重写基类中的以下几个虚函数：
- 类的构造函数和析构函数。
- :code:`init` 函数。用于初始化参数和设置。
- :code:`forward` 。实现网络层的前向传播。
- :code:`backward` 。实现网络层的后向传播。
- :code:`prefetch` 。用来从参数服务器预取参数矩阵相应的行。如果网络层不需要远程稀疏更新，则不需要重写该函数。（大多数网络层不需要支持远程稀疏更新）
头文件如下：
.. code-block:: c++
namespace paddle {
/**
* 全连接层的每个输出都连接到上一层的所有的神经元上。
* 它的输入与经过学习的参数做内积并加上偏置（可选）。
*
* 配置文件接口是fc_layer。
*/
class FullyConnectedLayer : public Layer {
protected:
WeightList weights_;
std::unique_ptr<Weight> biases_;
public:
explicit FullyConnectedLayer(const LayerConfig& config)
: Layer(config) {}
~FullyConnectedLayer() {}
bool init(const LayerMap& layerMap, const ParameterMap& parameterMap);
Weight& getWeight(int idx) { return *weights_[idx]; }
void prefetch();
void forward(PassType passType);
void backward(const UpdateCallback& callback = nullptr);
};
} // namespace paddle
头文件中把参数定义为类的成员变量。我们使用 :code:`Weight` 类作为参数的抽象，它支持多线程更新。该类的实现细节在“实现细节”中详细介绍。
- :code:`weights_` 是存有一系列变换矩阵的权重。在当前的实现方式下，网络层可以有多个输入。因此，它可能有不止一个权重。每个权重对应一个输入。
- :code:`biases_` 是存有偏置向量的权重。
全连接层没有网络层配置的超参数。如果一个网络层需要配置的话，通常的做法是将配置存于 :code:`LayerConfig& config` 中，并在类构建函数中把它放入一个类成员变量里。
下面的代码片段实现了 :code:`init` 函数。
- 首先，所有的 :code:`init` 函数必须先调用基类中的函数 :code:`Layer::init(layerMap, parameterMap);` 。该语句会为每个层初始化其所需要的变量和连接。
- 之后初始化所有的权重矩阵 :math:`W` 。当前的实现方式下，网络层可以有多个输入。因此，它可能有不止一个权重。
- 最后，初始化偏置向量。
.. code-block:: c++
bool FullyConnectedLayer::init(const LayerMap& layerMap,
const ParameterMap& parameterMap) {
/* 初始化父类 */
Layer::init(layerMap, parameterMap);
/* 初始化权重表 */
CHECK(inputLayers_.size() == parameters_.size());
for (size_t i = 0; i < inputLayers_.size(); i++) {
// 获得参数尺寸
size_t height = inputLayers_[i]->getSize();
size_t width = getSize();
// 新建一个权重
if (parameters_[i]->isSparse()) {
CHECK_LE(parameters_[i]->getSize(), width * height);
} else {
CHECK_EQ(parameters_[i]->getSize(), width * height);
}
Weight* w = new Weight(height, width, parameters_[i]);
// 将新建的权重加入权重表
weights_.emplace_back(w);
}
/* 初始化biases_ */
if (biasParameter_.get() != NULL) {
biases_ = std::unique_ptr<Weight>(new Weight(1, getSize(), biasParameter_));
}
return true;
}
实现前向传播的部分有下面几个步骤。
- 每个层在其 :code:`forward` 函数的开头必须调用 :code:`Layer::forward(passType);` 。
- 之后使用 :code:`reserveOutput(batchSize, size);` 为输出分配内存。由于我们支持训练数据有不同的批次大小，所以这一步是必要的。 :code:`reserveOutput` 会相应地改变输出的尺寸。为了保证效率，如果需要扩大矩阵，我们会重新分配内存；如果需要缩减矩阵，我们会继续使用现有的内存块。
- 之后使用矩阵运算函数来计算 :math:`\sum_i W_i x + b`。:code:`getInput(i).value` 返回第i个输入矩阵。每个输入都是一个 :math:`batchSize \times dim` 的矩阵，每行表示一个批次中的单个输入。对于我们支持的全部矩阵操作，请参考 :code:`paddle/math/Matrix.h`和:code:`paddle/math/BaseMatrix.h` 。
- 最终，使用 :code:`forwardActivation();` 进行激活操作。这会自动进行网络配置中声明的激活操作。
.. code-block:: c++
void FullyConnectedLayer::forward(PassType passType) {
Layer::forward(passType);
/* 若有必要，为output_申请内存 */
int batchSize = getInput(0).getBatchSize();
int size = getSize();
{
// 设置输出的尺寸
reserveOutput(batchSize, size);
}
MatrixPtr outV = getOutputValue();
// 对每个输入乘上变换矩阵
for (size_t i = 0; i != inputLayers_.size(); ++i) {
auto input = getInput(i);
CHECK(input.value) << "The input of 'fc' layer must be matrix";
i == 0 ? outV->mul(input.value, weights_[i]->getW(), 1, 0)
: outV->mul(input.value, weights_[i]->getW(), 1, 1);
}
/* 加上偏置向量 */
if (biases_.get() != NULL) {
outV->addBias(*(biases_->getW()), 1);
}
/* 激活 */ {
forwardActivation();
}
}
实现后向传播的部分有下面几个步骤。
- :code:`backwardActivation()` 计算激活函数的梯度。通过 :code:`getOutputGrad()` 来获得输出的梯度，调用该函数后，梯度会就地（不使用额外空间）乘上输出的梯度。
- 计算偏置的梯度。注意，我们使用 :code:`biases_->getWGrad()` 来得到某个特定参数的梯度矩阵。在一个参数的梯度被更新后，**必须**要调用 :code:`getParameterPtr()->incUpdate(callback);` 。这用于在多线程和多机上更新参数。
- 最后，计算转换矩阵和输入的梯度，并对相应的参数调用 :code:`incUpdate` 。PaddlePaddle可以通过该机制判断是否已经收集齐所有的梯度，从而可以做一些与计算重叠的工作（例如，网络通信）。
.. code-block:: c++
void FullyConnectedLayer::backward(const UpdateCallback& callback) {
/* 对激活求导 */ {
backwardActivation();
}
if (biases_ && biases_->getWGrad()) {
biases_->getWGrad()->collectBias(*getOutputGrad(), 1);
biases_->getParameterPtr()->incUpdate(callback);
}
bool syncFlag = hl_get_sync_flag();
for (size_t i = 0; i != inputLayers_.size(); ++i) {
/* 计算当前层权重的梯度 */
if (weights_[i]->getWGrad()) {
MatrixPtr input_T = getInputValue(i)->getTranspose();
MatrixPtr oGrad = getOutputGrad();
{
weights_[i]->getWGrad()->mul(input_T, oGrad, 1, 1);
}
}
/* 计算输入层的偏差 */
MatrixPtr preGrad = getInputGrad(i);
if (NULL != preGrad) {
MatrixPtr weights_T = weights_[i]->getW()->getTranspose();
preGrad->mul(getOutputGrad(), weights_T, 1, 1);
}
{
weights_[i]->getParameterPtr()->incUpdate(callback);
}
}
}
:code:`prefetch` 函数指出了在训练时需要从参数服务器取出的行。仅在远程稀疏训练时有效。使用远程稀疏方式训练时，完整的参数矩阵被分布在不同的参数服务器上。当网络层用一个批次做训练时，该批次的输入中仅有一个子集是非零的。因此，该层仅需要这些非零样本位置所对应的变换矩阵的那些行。 :code:`prefetch` 表明了这些行的标号。
大多数层不需要远程稀疏训练函数。这种情况下不需要重写该函数。
.. code-block:: c++
void FullyConnectedLayer::prefetch() {
for (size_t i = 0; i != inputLayers_.size(); ++i) {
auto* sparseParam =
dynamic_cast<SparsePrefetchRowCpuMatrix*>(weights_[i]->getW().get());
if (sparseParam) {
MatrixPtr input = getInputValue(i);
sparseParam->addRows(input);
}
}
}
最后，使用 :code:`REGISTER_LAYER(fc, FullyConnectedLayer);` 来注册该层。 :code:`fc` 是该层的标识符， :code:`FullyConnectedLayer` 是该层的类名。
.. code-block:: c++
namespace paddle {
REGISTER_LAYER(fc, FullyConnectedLayer);
}
若 :code:`cpp` 被放在 :code:`paddle/gserver/layers` 目录下，其会自动被加入编译列表。
写梯度检查单元测试
===============================
写梯度检查单元测试是一个验证新实现的层是否正确的相对简单的办法。梯度检查单元测试通过有限差分法来验证一个层的梯度。首先对输入做一个小的扰动 :math:`\Delta x` ，然后观察到输出的变化为 :math:`\Delta y` ，那么，梯度就可以通过这个方程计算得到 :math:`\frac{\Delta y}{\Delta x }` 。之后，再用这个梯度去和 :code:`backward` 函数得到的梯度去对比，以保证梯度计算的正确性。需要注意的是梯度检查仅仅验证了梯度的计算，并不保证 :code:`forward` 和 :code:`backward` 函数的实现是正确的。你需要一些更复杂的单元测试来保证你实现的网络层是正确的。
所有网络层的梯度检查单测都位于 :code:`paddle/gserver/tests/test_LayerGrad.cpp` 。我们建议你在写新网络层时把测试代码放入新的文件中。下面列出了全连接层的梯度检查单元测试。它包含以下几步：
+ 生成网络层配置。网络层配置包含以下几项：
- 偏置参数的大小。（例子中是4096）
- 层的类型。（例子中是fc）
- 层的大小。（例子中是4096）
- 激活的类型。（例子中是softmax）
- dropout的比例。（例子中是0.1）
+ 配置网络层的输入。在这个例子里，我们仅有一个输入。
- 输入的类型（ :code:`INPUT_DATA` ），可以是以下几种：
- :code:`INPUT_DATA` ：稠密向量。
- :code:`INPUT_LABEL` ：整数。
- :code:`INPUT_DATA_TARGET` ：稠密向量，但不用于计算梯度。
- :code:`INPUT_SEQUENCE_DATA` ：含有序列信息的稠密向量。
- :code:`INPUT_HASSUB_SEQUENCE_DATA` ：含有序列信息和子序列信息的稠密向量。
- :code:`INPUT_SEQUENCE_LABEL` ：含有序列信息的整数。
- :code:`INPUT_SPARSE_NON_VALUE_DATA` ：0-1稀疏数据。
- :code:`INPUT_SPARSE_FLOAT_VALUE_DATA` ：浮点稀疏数据。
- 输入的名字。（例子中是 :code:`layer_0` ）
- 输入的大小。（例子中是8192）
- 非零数字的个数，仅对稀疏数据有效。
- 稀疏数据的格式，仅对稀疏数据有效。
+ 对每个输入，都需要调用一次 :code:`config.layerConfig.add_inputs();` 。
+ 调用 :code:`testLayerGrad` 来做梯度检查。它包含以下参数。
- 层和输入的配置。（例子中是 :code:`config` ）
- 网络层的类型。（例子中是 :code:`fc` ）
- 梯度检查的输入数据的批次大小。（例子中是100）
- 输入是否是转置的。大多数层需要设置为 :code:`false` 。（例子中是 :code:`false` ）
- 是否使用权重。有些层或者激活需要做归一化以保证它们的输出的和是一个常数。例如，softmax激活的输出的和总是1。在这种情况下，我们不能通过常规的梯度检查的方式来计算梯度。因此我们采用输出的加权和（非常数）来计算梯度。（例子中是 :code:`true` ，因为全连接层的激活可以是softmax）
.. code-block:: c++
void testFcLayer(string format, size_t nnz) {
// Create layer configuration.
TestConfig config;
config.biasSize = 4096;
config.layerConfig.set_type("fc");
config.layerConfig.set_size(4096);
config.layerConfig.set_active_type("softmax");
config.layerConfig.set_drop_rate(0.1);
// Setup inputs.
config.inputDefs.push_back(
{INPUT_DATA, "layer_0", 8192, nnz, ParaSparse(format)});
config.layerConfig.add_inputs();
LOG(INFO) << config.inputDefs[0].sparse.sparse << " "
<< config.inputDefs[0].sparse.format;
for (auto useGpu : {false, true}) {
testLayerGrad(config, "fc", 100, /* trans */ false, useGpu,
/* weight */ true);
}
}
如果你要为了测试而增加新的文件，例如 :code:`paddle/gserver/tests/testFCGrad.cpp` ，你需要把该文件加入 :code:`paddle/gserver/tests/CMakeLists.txt` 中。下面给出了一个例子。当你执行命令 :code:`make tests` 时，所有的单测都会被执行一次。注意，有些层可能需要高精度来保证梯度检查单测正确执行。你需要在配置cmake时将 :code:`WITH_DOUBLE` 设置为 `ON` 。
.. code-block:: bash
add_unittest_without_exec(test_FCGrad
test_FCGrad.cpp
LayerGradUtil.cpp
TestUtil.cpp)
add_test(NAME test_FCGrad
COMMAND test_FCGrad)
实现python封装
========================
python封装的实现使得我们可以在配置文件中使用新实现的网络层。所有的python封装都在 :code:`python/paddle/trainer/config_parser.py` 中。全连接层python封装的例子中包含下面几步：
- 所有的Python封装都使用 :code:`@config_layer('fc')` 这样的装饰器。网络层的标识符为 :code:`fc` 。
- 实现构造函数 :code:`__init__` 。
- 它首先调用基构造函数 :code:`super(FCLayer, self).__init__(name, 'fc', size, inputs=inputs, **xargs)` 。 :code:`FCLayer` 是Python封装的类名。 :code:`fc` 是网络层的标识符。为了封装能够正确工作，这些名字必须要写对。
- 之后，计算变换矩阵的大小和格式（是否稀疏）。
.. code-block:: python
@config_layer('fc')
class FCLayer(LayerBase):
def __init__(
self,
name,
size,
inputs,
bias=True,
**xargs):
super(FCLayer, self).__init__(name, 'fc', size, inputs=inputs, **xargs)
for input_index in xrange(len(self.inputs)):
input_layer = self.get_input_layer(input_index)
psize = self.config.size * input_layer.size
dims = [input_layer.size, self.config.size]
format = self.inputs[input_index].format
sparse = format == "csr" or format == "csc"
if sparse:
psize = self.inputs[input_index].nnz
self.create_input_parameter(input_index, psize, dims, sparse, format)
self.create_bias_parameter(bias, self.config.size)
在网络配置中，网络层的细节可以通过下面这些代码片段来指定。这个类的参数包括：
- :code:`name` 是网络层实例的名字标识符。
- :code:`type` 是网络层的类型，通过网络层的标识符来指定。
- :code:`size` 是网络层输出的大小。
- :code:`bias` 表明这个层的一个实例是否需要偏置。
- :code:`inputs` 说明这个层的输入，输入是由一个list中的网络层实例的名字组成的。
.. code-block:: python
Layer(
name = "fc1",
type = "fc",
size = 64,
bias = True,
inputs = [Input("pool3")]
)
我们建议你为你的Python封装实现一个“助手”，使得搭模型时更方便。具体可以参考 :code:`python/paddle/trainer_config_helpers/layers.py` 。
doc/howto/dev/new_layer_en.rst
浏览文件 @ 106620ea
......@@ -209,7 +209,6 @@ The implementation of the backward part has the following steps.
if (biases_ && biases_->getWGrad()) {
biases_->getWGrad()->collectBias(*getOutputGrad(), 1);
/* Increasing the number of gradient */
biases_->getParameterPtr()->incUpdate(callback);
}
......@@ -297,7 +296,7 @@ All the gradient check unit tests are located in :code:`paddle/gserver/tests/tes
+ each inputs needs to call :code:`config.layerConfig.add_inputs();` once.
+ call :code:`testLayerGrad` to perform gradient checks. It has the following arguments.
- layer and input configurations. (:code:`config` in our example)
- type of the input. (:code:`fc` in our example)
- type of the layer. (:code:`fc` in our example)
- batch size of the gradient check. (100 in our example)
- whether the input is transpose. Most layers need to set it to :code:`false`. (:code:`false` in our example)
- whether to use weights. Some layers or activations perform normalization so that the sum of their output is a constant. For example, the sum of output of a softmax activation is one. In this case, we cannot correctly compute the gradients using regular gradient check techniques. A weighted sum of the output, which is not a constant, is utilized to compute the gradients. (:code:`true` in our example, because the activation of a fully connected layer can be softmax)
......@@ -310,7 +309,7 @@ All the gradient check unit tests are located in :code:`paddle/gserver/tests/tes
config.biasSize = 4096;
config.layerConfig.set_type("fc");
config.layerConfig.set_size(4096);
config.layerConfig.set_active_type("sigmoid");
config.layerConfig.set_active_type("softmax");
config.layerConfig.set_drop_rate(0.1);
// Setup inputs.
config.inputDefs.push_back(
......
doc/howto/index_cn.rst
浏览文件 @ 106620ea
......@@ -7,10 +7,11 @@
.. toctree::
:maxdepth: 1
usage/cmd_parameter/index_cn.rst
usage/concepts/use_concepts_cn.rst
usage/cluster/cluster_train_cn.md
usage/cluster/k8s/k8s_cn.md
usage/cluster/k8s/k8s_distributed_cn.md
usage/k8s/k8s_cn.md
usage/k8s/k8s_distributed_cn.md
开发标准
--------
......
doc/howto/index_en.rst
浏览文件 @ 106620ea
......@@ -7,8 +7,10 @@ Usage
.. toctree::
:maxdepth: 1
usage/cmd_parameter/index_en.md
usage/cmd_parameter/index_en.rst
usage/cluster/cluster_train_en.md
usage/k8s/k8s_en.md
usage/k8s/k8s_aws_en.md
Development
------------
......
doc/howto/usage/cmd_parameter/index_cn.rst 0 → 100644
浏览文件 @ 106620ea
.. _cmd_line_index:
设置命令行参数
===============
.. toctree::
:maxdepth: 1
use_case_cn.md
arguments_cn.md
detail_introduction_cn.md
doc/howto/usage/cmd_parameter/index_en.md 已删除 100644 → 0
浏览文件 @ 3fcd81fd
```eval_rst
.. _cmd_line_index:
```
# Set Command-line Parameters
* [Use Case](use_case_en.md)
* [Arguments](arguments_en.md)
* [Detailed Descriptions](detail_introduction_en.md)
doc/howto/usage/cmd_parameter/index_en.rst 0 → 100644
浏览文件 @ 106620ea
.. _cmd_line_index:
Set Command-line Parameters
===========================
.. toctree::
:maxdepth: 1
use_case_en.md
arguments_en.md
detail_introduction_en.md
doc/howto/usage/cluster/k8s-aws/README.md → doc/howto/usage/k8s/k8s_aws_en.md
浏览文件 @ 106620ea
# PaddlePaddle on AWS with Kubernetes
# Kubernetes on AWS
## Create AWS Account and IAM Account
......@@ -331,15 +331,15 @@ For sharing the training data across all the Kubernetes nodes, we use EFS (Elast
1. Make sure you added AmazonElasticFileSystemFullAccess policy in your group.
1. Create the Elastic File System in AWS console, and attach the new VPC with it.
<img src="create_efs.png" width="800">
<center>![](src/create_efs.png)</center>
1. Modify the Kubernetes security group under ec2/Security Groups, add additional inbound policy "All TCP TCP 0 - 65535 0.0.0.0/0" for Kubernetes default VPC security group.
<img src="add_security_group.png" width="800">
<center>![](src/add_security_group.png)</center>
1. Follow the EC2 mount instruction to mount the disk onto all the Kubernetes nodes, we recommend to mount EFS disk onto ~/efs.
<img src="efs_mount.png" width="800">
<center>![](src/efs_mount.png)</center>
Before starting the training, you should place your user config and divided training data onto EFS. When the training start, each task will copy related files from EFS into container, and it will also write the training results back onto EFS, we will show you how to place the data later in this article.
......
doc/howto/usage/cluster/k8s/k8s_cn.md → doc/howto/usage/k8s/k8s_cn.md
浏览文件 @ 106620ea
# Kubernetes 单机训练
# Kubernetes单机训练
在这篇文档里，我们介绍如何在 Kubernetes 集群上启动一个单机使用CPU的Paddle训练作业。在下一篇中，我们将介绍如何启动分布式训练作业。
......
doc/howto/usage/cluster/k8s/k8s_distributed_cn.md → doc/howto/usage/k8s/k8s_distributed_cn.md
浏览文件 @ 106620ea
# Kubernetes 分布式训练
# Kubernetes分布式训练
前一篇文章介绍了如何在Kubernetes集群上启动一个单机PaddlePaddle训练作业 (Job)。在这篇文章里，我们介绍如何在Kubernetes集群上进行分布式PaddlePaddle训练作业。关于PaddlePaddle的分布式训练，文章 [Cluster Training](https://github.com/baidu/Paddle/blob/develop/doc/cluster/opensource/cluster_train.md)介绍了一种通过SSH远程分发任务，进行分布式训练的方法，与此不同的是，本文将介绍在Kubernetes容器管理平台上快速构建PaddlePaddle容器集群，进行分布式训练的方案。
......@@ -22,7 +22,7 @@
首先，我们需要拥有一个Kubernetes集群，在这个集群中所有node与pod都可以互相通信。关于Kubernetes集群搭建，可以参考[官方文档](http://kubernetes.io/docs/getting-started-guides/kubeadm/)，在以后的文章中我们也会介绍AWS上搭建的方案。本文假设大家能找到几台物理机，并且可以按照官方文档在上面部署Kubernetes。在本文的环境中，Kubernetes集群中所有node都挂载了一个[MFS](http://moosefs.org/)（Moose filesystem，一种分布式文件系统）共享目录，我们通过这个目录来存放训练文件与最终输出的模型。关于MFS的安装部署，可以参考[MooseFS documentation](https://moosefs.com/documentation.html)。在训练之前，用户将配置与训练数据切分好放在MFS目录中，训练时，程序从此目录拷贝文件到容器内进行训练，将结果保存到此目录里。整体的结构图如下：
![paddle on kubernetes结构图](k8s-paddle-arch.png)
![paddle on kubernetes结构图](src/k8s-paddle-arch.png)
上图描述了一个3节点的分布式训练场景，Kubernetes集群的每个node上都挂载了一个MFS目录，这个目录可以通过volume的形式挂载到容器中。Kubernetes为这次训练创建了3个pod并且调度到了3个node上运行，每个pod包含一个PaddlePaddle容器。在容器创建后，会启动pserver与trainer进程，读取volume中的数据进行这次分布式训练。
......
doc/tutorials/gan/gan.png
查看替换文件 @ 3fcd81fd 浏览文件 @ 106620ea
doc/tutorials/gan/gan.png

32.5 KB | W: | H:

doc/tutorials/gan/gan.png

17.4 KB | W: | H:

doc/tutorials/gan/gan.png
doc/tutorials/gan/gan.png
doc/tutorials/gan/gan.png
doc/tutorials/gan/gan.png
  • 2-up
  • Swipe
  • Onion skin
doc/tutorials/gan/index_en.md
浏览文件 @ 106620ea
......@@ -4,9 +4,7 @@ This demo implements GAN training described in the original [GAN paper](https://
The high-level structure of GAN is shown in Figure. 1 below. It is composed of two major parts: a generator and a discriminator, both of which are based on neural networks. The generator takes in some kind of noise with a known distribution and transforms it into an image. The discriminator takes in an image and determines whether it is artificially generated by the generator or a real image. So the generator and the discriminator are in a competitive game in which generator is trying to generate image to look as real as possible to fool the discriminator, while the discriminator is trying to distinguish between real and fake images.
<p align="center">
<img src="./gan.png" width="500" height="300">
</p>
<center>![](./gan.png)</center>
<p align="center">
Figure 1. GAN-Model-Structure
<a href="https://ishmaelbelghazi.github.io/ALI/">figure credit</a>
......@@ -111,9 +109,7 @@ $python gan_trainer.py -d uniform --useGpu 1
```
The generated samples can be found in ./uniform_samples/ and one example is shown below as Figure 2. One can see that it roughly recovers the 2D uniform distribution.
<p align="center">
<img src="./uniform_sample.png" width="300" height="300">
</p>
<center>![](./uniform_sample.png)</center>
<p align="center">
Figure 2. Uniform Sample
</p>
......@@ -135,9 +131,7 @@ To train the GAN model on mnist data, one can use the following command:
$python gan_trainer.py -d mnist --useGpu 1
```
The generated sample images can be found at ./mnist_samples/ and one example is shown below as Figure 3.
<p align="center">
<img src="./mnist_sample.png" width="300" height="300">
</p>
<center>![](./mnist_sample.png)</center>
<p align="center">
Figure 3. MNIST Sample
</p>
doc/tutorials/gan/uniform_sample.png
查看替换文件 @ 3fcd81fd 浏览文件 @ 106620ea
doc/tutorials/gan/uniform_sample.png

20.1 KB | W: | H:

doc/tutorials/gan/uniform_sample.png

24.3 KB | W: | H:

doc/tutorials/gan/uniform_sample.png
doc/tutorials/gan/uniform_sample.png
doc/tutorials/gan/uniform_sample.png
doc/tutorials/gan/uniform_sample.png
  • 2-up
  • Swipe
  • Onion skin
doc/tutorials/index_cn.md
浏览文件 @ 106620ea
......@@ -2,6 +2,7 @@
* [快速入门](quick_start/index_cn.rst)
* [个性化推荐](rec/ml_regression_cn.rst)
* [图像分类](image_classification/index_cn.md)
* [情感分析](sentiment_analysis/index_cn.md)
* [语义角色标注](semantic_role_labeling/index_cn.md)
* [机器翻译](text_generation/index_cn.md)
......@@ -9,3 +10,4 @@
## 常用模型
* [ResNet模型](imagenet_model/resnet_model_cn.md)
* [词向量模型](embedding_model/index_cn.md)
doc/tutorials/index_en.md
浏览文件 @ 106620ea
......@@ -7,6 +7,7 @@ There are several examples and demos here.
* [Sentiment Analysis](sentiment_analysis/index_en.md)
* [Semantic Role Labeling](semantic_role_labeling/index_en.md)
* [Text Generation](text_generation/index_en.md)
* [Image Auto-Generation](gan/index_en.md)
## Model Zoo
* [ImageNet: ResNet](imagenet_model/resnet_model_en.md)
......
paddle/api/paddle_ld_flags.py
浏览文件 @ 106620ea
......@@ -50,7 +50,6 @@ try:
self.glog_libs = LIBGLOG_LIBRARY
self.with_coverage = PaddleLDFlag.cmake_bool(WITH_COVERALLS)
self.gflags_libs = GFLAGS_LIBRARIES
self.gflags_location = GFLAGS_LOCATION
self.cblas_libs = CBLAS_LIBRARIES
self.curt = CUDA_LIBRARIES
......@@ -88,7 +87,7 @@ try:
"-lpaddle_api",
self.normalize_flag(self.protolib),
self.normalize_flag(self.glog_libs),
self.normalize_flag(self.gflags_libs),
self.normalize_flag("gflags"),
self.normalize_flag(self.zlib),
self.normalize_flag(self.thread),
self.normalize_flag(self.dl_libs),
......@@ -114,10 +113,7 @@ try:
return cmake_flag
elif cmake_flag.startswith("-l"): # normal link command
return cmake_flag
elif cmake_flag in [
"gflags-shared", "gflags-static", "gflags_nothreads-shared",
"gflags_nothreads-static"
]: # special for gflags
elif cmake_flag == "gflags": # special for gflags
assert PaddleLDFlag.cmake_bool(self.gflags_location)
return self.gflags_location
elif len(cmake_flag) != 0:
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或