Merge branch 'develop' of github.com:apache/rocketmq into develop

CONTRIBUTING.md

@@ -10,7 +10,7 @@ Nor is code the only way to contribute to the project. We strongly value documen
 To submit a change for inclusion, please do the following:
 
 #### If the change is non-trivial please include some unit tests that cover the new functionality.
-#### If you are introducing a completely new feature or API it is a good idea to start a wiki and get consensus on the basic design first.
+#### If you are introducing a completely new feature or API it is a good idea to start a [RIP](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal) and get consensus on the basic design first.
 #### It is our job to follow up on patches in a timely fashion. Nag us if we aren't doing our job (sometimes we drop things).
 
 ## Becoming a Committer
@@ -19,9 +19,8 @@ We are always interested in adding new contributors. What we look for are series
 
 Nowadays,we have several important contribution points:
 #### Wiki & JavaDoc
-#### RocketMQ Console
 #### RocketMQ SDK(C++\.Net\Php\Python\Go\Node.js)
-#### RocketMQ MySQL(Oracle\PostgreSQL\Redis\MongoDB\HBase\MSSQL) Replicator
+#### RocketMQ Connectors
 
 ##### Prerequisite
 If you want to contribute the above listing points, you must abide our some prerequisites:

README.md

@@ -51,3 +51,23 @@ We always welcome new contributions, whether for trivial cleanups, [big new feat
 [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html) Copyright (C) Apache Software Foundation
 
 
+----------
+## Export Control Notice
+This distribution includes cryptographic software. The country in which you currently reside may have
+restrictions on the import, possession, use, and/or re-export to another country, of encryption software.
+BEFORE using any encryption software, please check your country's laws, regulations and policies concerning
+the import, possession, or use, and re-export of encryption software, to see if this is permitted. See
+<http://www.wassenaar.org/> for more information.
+
+The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this
+software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software
+using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache
+Software Foundation distribution makes it eligible for export under the License Exception ENC Technology
+Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for
+both object code and source code.
+
+The following provides more details on the included cryptographic software:
+
+This software uses Apache Commons Crypto (https://commons.apache.org/proper/commons-crypto/) to
+support authentication, and encryption and decryption of data sent across the network between
+services.

distribution/benchmark/runclass.sh

@@ -24,9 +24,37 @@ fi
 BASE_DIR=$(dirname $0)/..
 CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
 
+# The RAMDisk initializing size in MB on Darwin OS for gc-log
+DIR_SIZE_IN_MB=600
+
+choose_gc_log_directory()
+{
+    case "`uname`" in
+        Darwin)
+            if [ ! -d "/Volumes/RAMDisk" ]; then
+                # create ram disk on Darwin systems as gc-log directory
+                DEV=`hdiutil attach -nomount ram://$((2 * 1024 * DIR_SIZE_IN_MB))` > /dev/null
+                diskutil eraseVolume HFS+ RAMDisk ${DEV} > /dev/null
+                echo "Create RAMDisk /Volumes/RAMDisk for gc logging on Darwin OS."
+            fi
+            GC_LOG_DIR="/Volumes/RAMDisk"
+        ;;
+        *)
+            # check if /dev/shm exists on other systems
+            if [ -d "/dev/shm" ]; then
+                GC_LOG_DIR="/dev/shm"
+            else
+                GC_LOG_DIR=${BASE_DIR}
+            fi
+        ;;
+    esac
+}
+
+choose_gc_log_directory
+
 JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn256m -XX:PermSize=128m -XX:MaxPermSize=320m"
 JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:+DisableExplicitGC"
-JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
+JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:${GC_LOG_DIR}/rmq_run_class_gc_%p_%t.log -XX:+PrintGCDetails"
 JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
 JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${BASE_DIR}/lib"
 JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"

distribution/bin/runbroker.sh

@@ -36,9 +36,37 @@ export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
 #===========================================================================================
 # JVM Configuration
 #===========================================================================================
+# The RAMDisk initializing size in MB on Darwin OS for gc-log
+DIR_SIZE_IN_MB=600
+
+choose_gc_log_directory()
+{
+    case "`uname`" in
+        Darwin)
+            if [ ! -d "/Volumes/RAMDisk" ]; then
+                # create ram disk on Darwin systems as gc-log directory
+                DEV=`hdiutil attach -nomount ram://$((2 * 1024 * DIR_SIZE_IN_MB))` > /dev/null
+                diskutil eraseVolume HFS+ RAMDisk ${DEV} > /dev/null
+                echo "Create RAMDisk /Volumes/RAMDisk for gc logging on Darwin OS."
+            fi
+            GC_LOG_DIR="/Volumes/RAMDisk"
+        ;;
+        *)
+            # check if /dev/shm exists on other systems
+            if [ -d "/dev/shm" ]; then
+                GC_LOG_DIR="/dev/shm"
+            else
+                GC_LOG_DIR=${BASE_DIR}
+            fi
+        ;;
+    esac
+}
+
+choose_gc_log_directory
+
 JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
 JAVA_OPT="${JAVA_OPT} -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0"
-JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
+JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:${GC_LOG_DIR}/rmq_broker_gc_%p_%t.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
 JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
 JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
 JAVA_OPT="${JAVA_OPT} -XX:+AlwaysPreTouch"

distribution/bin/runserver.sh

@@ -36,9 +36,38 @@ export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}
 #===========================================================================================
 # JVM Configuration
 #===========================================================================================
+# The RAMDisk initializing size in MB on Darwin OS for gc-log
+DIR_SIZE_IN_MB=600
+
+choose_gc_log_directory()
+{
+    case "`uname`" in
+        Darwin)
+            if [ ! -d "/Volumes/RAMDisk" ]; then
+                # create ram disk on Darwin systems as gc-log directory
+                DEV=`hdiutil attach -nomount ram://$((2 * 1024 * DIR_SIZE_IN_MB))` > /dev/null
+                diskutil eraseVolume HFS+ RAMDisk ${DEV} > /dev/null
+                echo "Create RAMDisk /Volumes/RAMDisk for gc logging on Darwin OS."
+            fi
+            GC_LOG_DIR="/Volumes/RAMDisk"
+        ;;
+        *)
+            # check if /dev/shm exists on other systems
+            if [ -d "/dev/shm" ]; then
+                GC_LOG_DIR="/dev/shm"
+            else
+                GC_LOG_DIR=${BASE_DIR}
+            fi
+        ;;
+    esac
+}
+
+choose_gc_log_directory
+
 JAVA_OPT="${JAVA_OPT} -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
 JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8  -XX:-UseParNewGC"
-JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
+JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:${GC_LOG_DIR}/rmq_srv_gc_%p_%t.log -XX:+PrintGCDetails"
+JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
 JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
 JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
 JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"

docs/cn/README.md

-Apache RocketMQ开发者指南
+Apache RocketMQ开发者指南
 --------
 
 ##### 这个开发者指南是帮忙您快速了解,并使用 Apache RocketMQ
@@ -14,7 +14,7 @@ Apache RocketMQ开发者指南
 
 - [架构(Architecture)](architecture.md)：介绍RocketMQ部署架构和技术架构。
 
-- [设计(Design)](design.md)：介绍RocketMQ关键机制的设计原理，主要包括消息存储、通信机制、消息过滤、负载均衡、事物消息等。
+- [设计(Design)](design.md)：介绍RocketMQ关键机制的设计原理，主要包括消息存储、通信机制、消息过滤、负载均衡、事务消息等。
 
 
 ### 3. 样例

docs/cn/features.md

@@ -12,7 +12,7 @@
 - 分区顺序
 对于指定的一个 Topic，所有消息根据 sharding key 进行区块分区。 同一个分区内的消息按照严格的 FIFO 顺序进行发布和消费。 Sharding key 是顺序消息中用来区分不同分区的关键字段，和普通消息的 Key 是完全不同的概念。
 适用场景：性能要求高，以 sharding key 作为分区字段，在同一个区块中严格的按照 FIFO 原则进行消息发布和消费的场景。
-##3 消息过滤
+## 3 消息过滤
 RocketMQ的消费者可以根据Tag进行消息过滤，也支持自定义属性过滤。消息过滤目前是在Broker端实现的，优点是减少了对于Consumer无用消息的网络传输，缺点是增加了Broker的负担、而且实现相对复杂。
 ## 4 消息可靠性
 RocketMQ支持消息的高可靠，影响消息可靠性的几种情况：