提交 1d254347 编写于 作者: G gaojun2048 提交者: gaojun2048

[Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and...

[Feature-3189][alert,spi,dao,plugin-api] base code of dolphinscheduler spi and alert plugin implement (#3601)

* DS SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* Add DolphinScheduler SPI , and rebuilt the code of the Alert plug-in based on SPI

* add TODO

* delete

* compile

* spi commit

* Plugin Alert

* fix some bug

* add todo

* change web ui from alpacajs to form-create

* remove module

* add plugin schema

* add license header

* update alert and spi module version

* update the alert plugin sub module version

* comment the maven.local.repository param

* move utils from spi to common module

* add license header

* add license header and delete some chinese comment

* update spi packages

* delete no use alert_xx.properties

* update mysql.connector.version back to 5.1.34

* delete no use comment in pom.xml

* update email stmp password

* add license

* add semicolon to sql/upgrade/1.4.0_schema/mysql/dolphinscheduler_ddl.sql file

* format the code style

* format new clase file with checkstyle

* update plugin params to Builder model

* move JSONUtils to SPI because plugin can not dependency common module

* move JSONUtils to SPI because plugin can not dependency common module

* delete collection dependency

* replace PluginParamsTransfer to spi PluginParamsTransfer

* update dolphinscheduler-maven-plugin to 1.0.0

* update license

* update apache-rat-plugin add exclude '.iml' file

* check license

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* ArtifactResolver only use in development and configPlugins is not empty

* default datasource should be postgresql

* add license files

* add license files

* postgresql port should be 5432

* postgresql test

* mv show_type to spi

add license header to AlertConstants

* check style fix

* copy check style file from branch dev

* alert show_type set by plugin

* alert show_type set by plugin

* add PluginDefineMapper to dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/registry/DependencyConfig.java

* add Bean to TaskCallbackServiceTestConfig

* add Bean to TaskCallbackServiceTestConfig

* fix check style

* check style fix

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* fix check style

* [Feature-3682][ui]Add form-create plug-in and alarm group management add sample demo (#3683)

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* check style fix

* rollback test change

* rollback test change

* rollback dao pom change

* [feature-3665][ui]Add element-ui (#3666)

* [feature-3665][ui]Add element-ui

* add license

* Add form-create plug-in and alarm group management add sample demo

* Modify node version

* fix

* fix

* add ut to pom.xml

* add upgrade schema to global schema

* fix ut failed

* fix ut failed

* fix ut failed

* fix ut failed

* add test EmailAlertPluginTest to pom.xml

* fix ut failed

* fix ut failed

* fix check style

* update license header to presto license header

* presto license header not check

* fix ut coverage

* fix ut coverage

* fix ut

* fix ut

* fix ut

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage

* fix ut coverage
Co-authored-by: Nbreak60 <790061044@qq.com>
上级 1618b98c
......@@ -216,3 +216,7 @@ The text of each license is the standard Apache 2.0 license.
ScriptRunner from https://github.com/mybatis/mybatis-3 Apache 2.0
mvnw files from https://github.com/takari/maven-wrapper Apache 2.0
PropertyPlaceholderHelper from https://github.com/spring-projects/spring-framework Apache 2.0
DolphinPluginClassLoader from https://github.com/prestosql/presto Apache 2.0
DolphinPluginDiscovery from https://github.com/prestosql/presto Apache 2.0
DolphinPluginLoader from https://github.com/prestosql/presto Apache 2.0
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert-dingtalk</artifactId>
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert-email</artifactId>
<!-- can be load as a Alert Plugin when development and run server in IDE -->
<packaging>dolphinscheduler-plugin</packaging>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-email</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<type>jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
\ No newline at end of file
......@@ -14,70 +14,56 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.utils;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.junit.Assert.*;
public class PropertyUtilsTest {
private static final Logger logger = LoggerFactory.getLogger(PropertyUtilsTest.class);
/**
* Test getString
*/
@Test
public void testGetString() {
String result = PropertyUtils.getString("test.string");
logger.info(result);
assertEquals("teststring", result);
//If key is null, then return null
result = PropertyUtils.getString(null);
assertNull(result);
}
package org.apache.dolphinscheduler.plugin.alert.email;
/**
* Test getBoolean
*/
@Test
public void testGetBoolean() {
import org.apache.dolphinscheduler.spi.alert.AlertChannel;
import org.apache.dolphinscheduler.spi.alert.AlertData;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import org.apache.dolphinscheduler.spi.alert.AlertResult;
import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer;
//Expected true
Boolean result = PropertyUtils.getBoolean("test.true");
assertTrue(result);
import java.util.Map;
//Expected false
result = PropertyUtils.getBoolean("test.false");
assertFalse(result);
}
/**
* Test getLong
*/
@Test
public void testGetLong() {
long result = PropertyUtils.getLong("test.long");
assertSame(100L, result);
}
/**
* Test getDouble
*/
@Test
public void testGetDouble() {
//If key is undefine in alert.properties, and there is a defaultval, then return defaultval
double result = PropertyUtils.getDouble("abc", 5.0);
assertEquals(5.0, result, 0);
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
result = PropertyUtils.getDouble("cba", 5.0);
assertEquals(3.1, result, 0.01);
/**
* email alert channel . use email to seed the alertInfo
*/
public class EmailAlertChannel implements AlertChannel {
private static final Logger logger = LoggerFactory.getLogger(EmailAlertChannel.class);
@Override
public AlertResult process(AlertInfo info) {
AlertData alert = info.getAlertData();
String alertParams = info.getAlertParams();
Map<String, String> paramsMap = PluginParamsTransfer.getPluginParamsMap(alertParams);
MailSender mailSender = new MailSender(paramsMap);
AlertResult alertResult = mailSender.sendMails(alert.getTitle(), alert.getContent());
//send flag
boolean flag = false;
if (alertResult == null) {
alertResult = new AlertResult();
alertResult.setStatus("false");
alertResult.setMessage("alert send error.");
logger.info("alert send error : {}", alertResult.getMessage());
return alertResult;
}
flag = Boolean.parseBoolean(String.valueOf(alertResult.getStatus()));
if (flag) {
logger.info("alert send success");
alertResult.setMessage("email send success.");
} else {
alertResult.setMessage("alert send error.");
logger.info("alert send error : {}", alertResult.getMessage());
}
return alertResult;
}
}
\ No newline at end of file
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
import org.apache.dolphinscheduler.spi.alert.AlertChannel;
import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory;
import org.apache.dolphinscheduler.spi.alert.AlertConstants;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.params.InputParam;
import org.apache.dolphinscheduler.spi.params.PasswordParam;
import org.apache.dolphinscheduler.spi.params.RadioParam;
import org.apache.dolphinscheduler.spi.params.base.DataType;
import org.apache.dolphinscheduler.spi.params.base.ParamsOptions;
import org.apache.dolphinscheduler.spi.params.base.PluginParams;
import org.apache.dolphinscheduler.spi.params.base.Validate;
import java.util.ArrayList;
import java.util.List;
/**
* email alert factory
*/
public class EmailAlertChannelFactory implements AlertChannelFactory {
@Override
public String getName() {
return "email alert";
}
@Override
public List<PluginParams> getParams() {
List<PluginParams> paramsList = new ArrayList<>();
InputParam receivesParam = InputParam.newBuilder(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS, MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERS)
.setPlaceholder("please input receives")
.addValidate(Validate.newBuilder()
.setRequired(true)
.build())
.build();
InputParam receiveCcsParam = InputParam.newBuilder(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS, MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERCCS)
.build();
InputParam mailSmtpHost = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_HOST, MailParamsConstants.MAIL_SMTP_HOST)
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
InputParam mailSmtpPort = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_PORT, MailParamsConstants.MAIL_SMTP_PORT)
.setValue(25)
.addValidate(Validate.newBuilder()
.setRequired(true)
.setType(DataType.NUMBER.getDataType())
.build())
.build();
InputParam mailSender = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SENDER, MailParamsConstants.MAIL_SENDER)
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
RadioParam enableSmtpAuth = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_AUTH, MailParamsConstants.MAIL_SMTP_AUTH)
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.setValue(true)
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
InputParam mailUser = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_USER, MailParamsConstants.MAIL_USER)
.setPlaceholder("if enable use authentication, you need input user")
.build();
PasswordParam mailPassword = PasswordParam.newBuilder(MailParamsConstants.NAME_MAIL_PASSWD, MailParamsConstants.MAIL_PASSWD)
.setPlaceholder("if enable use authentication, you need input password")
.build();
RadioParam enableTls = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE, MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE)
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.setValue(false)
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
RadioParam enableSsl = RadioParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE, MailParamsConstants.MAIL_SMTP_SSL_ENABLE)
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.setValue(false)
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
InputParam sslTrust = InputParam.newBuilder(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST, MailParamsConstants.MAIL_SMTP_SSL_TRUST)
.setValue("*")
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, AlertConstants.SHOW_TYPE)
.addParamsOptions(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false))
.addParamsOptions(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false))
.addParamsOptions(new ParamsOptions(ShowType.ATTACHMENT.getDescp(), ShowType.ATTACHMENT.getDescp(), false))
.addParamsOptions(new ParamsOptions(ShowType.TABLEATTACHMENT.getDescp(), ShowType.TABLEATTACHMENT.getDescp(), false))
.setValue(ShowType.TABLE.getDescp())
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
paramsList.add(receivesParam);
paramsList.add(receiveCcsParam);
paramsList.add(mailSmtpHost);
paramsList.add(mailSmtpPort);
paramsList.add(mailSender);
paramsList.add(enableSmtpAuth);
paramsList.add(mailUser);
paramsList.add(mailPassword);
paramsList.add(enableTls);
paramsList.add(enableSsl);
paramsList.add(sslTrust);
paramsList.add(showType);
return paramsList;
}
@Override
public AlertChannel create() {
return new EmailAlertChannel();
}
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin;
import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory;
import com.google.common.collect.ImmutableList;
/**
* email alert plugin
*/
public class EmailAlertPlugin implements DolphinSchedulerPlugin {
@Override
public Iterable<AlertChannelFactory> getAlertChannelFactorys() {
return ImmutableList.of(new EmailAlertChannelFactory());
}
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
public class EmailConstants {
public static final String XLS_FILE_PATH = "xls.file.path";
public static final String MAIL_TRANSPORT_PROTOCOL = "mail.transport.protocol";
public static final String DEFAULT_SMTP_PORT = "25";
public static final String TEXT_HTML_CHARSET_UTF_8 = "text/html;charset=utf-8";
public static final int NUMBER_1000 = 1000;
public static final String TR = "<tr>";
public static final String TD = "<td>";
public static final String TD_END = "</td>";
public static final String TR_END = "</tr>";
public static final String TITLE = "title";
public static final String CONTENT = "content";
public static final String TH = "<th>";
public static final String TH_END = "</th>";
public static final String MARKDOWN_QUOTE = ">";
public static final String MARKDOWN_ENTER = "\n";
public static final String HTML_HEADER_PREFIX = new StringBuilder("<!DOCTYPE HTML PUBLIC '-//W3C//DTD HTML 4.01 Transitional//EN' 'http://www.w3.org/TR/html4/loose.dtd'>")
.append("<html>")
.append("<head>")
.append("<title>dolphinscheduler</title>")
.append("<meta name='Keywords' content=''>")
.append("<meta name='Description' content=''>")
.append("<style type=\"text/css\">")
.append("table {margin-top:0px;padding-top:0px;border:1px solid;font-size: 14px;color: #333333;border-width: 1px;border-color: #666666;border-collapse: collapse;}")
.append("table th {border-width: 1px;padding: 8px;border-style: solid;border-color: #666666;background-color: #dedede;text-align: left;}")
.append("table td {border-width: 1px;padding: 8px;border-style: solid;border-color: #666666;background-color: #ffffff;text-align: left;}")
.append("</style>")
.append("</head>")
.append("<body style=\"margin:0;padding:0\"><table border=\"1px\" cellpadding=\"5px\" cellspacing=\"-10px\"> ")
.toString();
public static final String TABLE_BODY_HTML_TAIL = "</table></body></html>";
public static final String UTF_8 = "UTF-8";
public static final String EXCEL_SUFFIX_XLS = ".xls";
public static final String SINGLE_SLASH = "/";
}
......@@ -14,23 +14,30 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
import org.apache.dolphinscheduler.common.utils.CollectionUtils;
package org.apache.dolphinscheduler.plugin.alert.email;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.poi.hssf.usermodel.HSSFCell;
import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.hssf.usermodel.HSSFSheet;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.HorizontalAlignment;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.*;
import org.apache.dolphinscheduler.common.utils.*;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* excel utils
......@@ -38,19 +45,21 @@ import org.apache.dolphinscheduler.common.utils.*;
public class ExcelUtils {
private static final Logger logger = LoggerFactory.getLogger(ExcelUtils.class);
/**
* generate excel file
* @param content the content
* @param title the title
*
* @param content the content
* @param title the title
* @param xlsFilePath the xls path
*/
public static void genExcelFile(String content,String title,String xlsFilePath){
public static void genExcelFile(String content, String title, String xlsFilePath) {
List<LinkedHashMap> itemsList;
//The JSONUtils.toList has been try catch ex
itemsList = JSONUtils.toList(content, LinkedHashMap.class);
if (CollectionUtils.isEmpty(itemsList)){
if (CollectionUtils.isEmpty(itemsList)) {
logger.error("itemsList is null");
throw new RuntimeException("itemsList is null");
}
......@@ -60,81 +69,81 @@ public class ExcelUtils {
List<String> headerList = new ArrayList<>();
Iterator<Map.Entry<String, Object>> iter = headerMap.entrySet().iterator();
while (iter.hasNext()){
while (iter.hasNext()) {
Map.Entry<String, Object> en = iter.next();
headerList.add(en.getKey());
}
HSSFWorkbook wb = null;
FileOutputStream fos = null;
try {
// declare a workbook
wb = new HSSFWorkbook();
// generate a table
HSSFSheet sheet = wb.createSheet();
HSSFRow row = sheet.createRow(0);
//set the height of the first line
row.setHeight((short)500);
//set Horizontal right
CellStyle cellStyle = wb.createCellStyle();
cellStyle.setAlignment(HorizontalAlignment.RIGHT);
//setting excel headers
for (int i = 0; i < headerList.size(); i++) {
HSSFCell cell = row.createCell(i);
cell.setCellStyle(cellStyle);
cell.setCellValue(headerList.get(i));
}
//setting excel body
int rowIndex = 1;
for (LinkedHashMap<String, Object> itemsMap : itemsList){
Object[] values = itemsMap.values().toArray();
row = sheet.createRow(rowIndex);
//setting excel body height
row.setHeight((short)500);
rowIndex++;
for (int j = 0 ; j < values.length ; j++){
HSSFCell cell1 = row.createCell(j);
cell1.setCellStyle(cellStyle);
cell1.setCellValue(String.valueOf(values[j]));
}
}
for (int i = 0; i < headerList.size(); i++) {
sheet.setColumnWidth(i, headerList.get(i).length() * 800);
}
File file = new File(xlsFilePath);
if (!file.exists()) {
file.mkdirs();
}
//setting file output
fos = new FileOutputStream(xlsFilePath + Constants.SINGLE_SLASH + title + Constants.EXCEL_SUFFIX_XLS);
wb.write(fos);
}catch (Exception e){
logger.error("generate excel error",e);
throw new RuntimeException("generate excel error",e);
}finally {
if (wb != null){
try {
wb.close();
} catch (IOException e) {
logger.error(e.getMessage(),e);
}
}
if (fos != null){
try {
fos.close();
} catch (IOException e) {
logger.error(e.getMessage(),e);
}
}
}
try {
// declare a workbook
wb = new HSSFWorkbook();
// generate a table
HSSFSheet sheet = wb.createSheet();
HSSFRow row = sheet.createRow(0);
//set the height of the first line
row.setHeight((short) 500);
//set Horizontal right
CellStyle cellStyle = wb.createCellStyle();
cellStyle.setAlignment(HorizontalAlignment.RIGHT);
//setting excel headers
for (int i = 0; i < headerList.size(); i++) {
HSSFCell cell = row.createCell(i);
cell.setCellStyle(cellStyle);
cell.setCellValue(headerList.get(i));
}
//setting excel body
int rowIndex = 1;
for (LinkedHashMap<String, Object> itemsMap : itemsList) {
Object[] values = itemsMap.values().toArray();
row = sheet.createRow(rowIndex);
//setting excel body height
row.setHeight((short) 500);
rowIndex++;
for (int j = 0; j < values.length; j++) {
HSSFCell cell1 = row.createCell(j);
cell1.setCellStyle(cellStyle);
cell1.setCellValue(String.valueOf(values[j]));
}
}
for (int i = 0; i < headerList.size(); i++) {
sheet.setColumnWidth(i, headerList.get(i).length() * 800);
}
File file = new File(xlsFilePath);
if (!file.exists()) {
file.mkdirs();
}
//setting file output
fos = new FileOutputStream(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS);
wb.write(fos);
} catch (Exception e) {
logger.error("generate excel error", e);
throw new RuntimeException("generate excel error", e);
} finally {
if (wb != null) {
try {
wb.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
}
}
}
}
\ No newline at end of file
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
/**
* mail plugin params json use
*/
public class MailParamsConstants {
public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERS = "receivers";
public static final String NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS = "receivers";
public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "receiverCcs";
public static final String NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "receiverCcs";
public static final String MAIL_PROTOCOL = "mail.transport.protocol";
public static final String NAME_MAIL_PROTOCOL = "mailProtocol";
public static final String MAIL_SMTP_HOST = "mail.smtp.host";
public static final String NAME_MAIL_SMTP_HOST = "mailServerHost";
public static final String MAIL_SMTP_PORT = "mail.smtp.port";
public static final String NAME_MAIL_SMTP_PORT = "mailServerPort";
public static final String MAIL_SENDER = "mail.sender";
public static final String NAME_MAIL_SENDER = "mailSender";
public static final String MAIL_SMTP_AUTH = "mail.smtp.auth";
public static final String NAME_MAIL_SMTP_AUTH = "enableSmtpAuth";
public static final String MAIL_USER = "mail.user";
public static final String NAME_MAIL_USER = "mailUser";
public static final String MAIL_PASSWD = "mail.passwd";
public static final String NAME_MAIL_PASSWD = "mailPasswd";
public static final String MAIL_SMTP_STARTTLS_ENABLE = "mail.smtp.starttls.enable";
public static final String NAME_MAIL_SMTP_STARTTLS_ENABLE = "starttlsEnable";
public static final String MAIL_SMTP_SSL_ENABLE = "mail.smtp.ssl.enable";
public static final String NAME_MAIL_SMTP_SSL_ENABLE = "sslEnable";
public static final String MAIL_SMTP_SSL_TRUST = "mail.smtp.ssl.trust";
public static final String NAME_MAIL_SMTP_SSL_TRUST = "mailSmtpSslTrust";
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
import static java.util.Objects.requireNonNull;
import org.apache.dolphinscheduler.plugin.alert.email.template.AlertTemplate;
import org.apache.dolphinscheduler.plugin.alert.email.template.DefaultHTMLTemplate;
import org.apache.dolphinscheduler.spi.alert.AlertConstants;
import org.apache.dolphinscheduler.spi.alert.AlertResult;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.mail.EmailException;
import org.apache.commons.mail.HtmlEmail;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import javax.mail.Authenticator;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.PasswordAuthentication;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeBodyPart;
import javax.mail.internet.MimeMessage;
import javax.mail.internet.MimeMultipart;
import javax.mail.internet.MimeUtility;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.sun.mail.smtp.SMTPProvider;
/**
* mail utils
*/
public class MailSender {
public static final Logger logger = LoggerFactory.getLogger(MailSender.class);
private List<String> receivers;
private List<String> receiverCcs;
private String mailProtocol = "SMTP";
private String mailSmtpHost;
private String mailSmtpPort;
private String mailSender;
private String enableSmtpAuth;
private String mailUser;
private String mailPasswd;
private String mailUseStartTLS;
private String mailUseSSL;
private String xlsFilePath;
private String sslTrust;
private String showType;
private AlertTemplate alertTemplate;
public MailSender(Map<String, String> config) {
String receiversConfig = config.get(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS);
if (receiversConfig == null || "".equals(receiversConfig)) {
throw new RuntimeException(MailParamsConstants.PLUGIN_DEFAULT_EMAIL_RECEIVERS + "must not be null");
}
receivers = Arrays.asList(receiversConfig.split(","));
String receiverCcsConfig = config.get(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS);
receiverCcs = new ArrayList<>();
if (receiverCcsConfig != null && !"".equals(receiverCcsConfig)) {
receiverCcs = Arrays.asList(receiverCcsConfig.split(","));
}
mailSmtpHost = config.get(MailParamsConstants.NAME_MAIL_SMTP_HOST);
requireNonNull(mailSmtpHost, MailParamsConstants.MAIL_SMTP_HOST + " must not null");
mailSmtpPort = config.get(MailParamsConstants.NAME_MAIL_SMTP_PORT);
requireNonNull(mailSmtpPort, MailParamsConstants.MAIL_SMTP_PORT + " must not null");
mailSender = config.get(MailParamsConstants.NAME_MAIL_SENDER);
requireNonNull(mailSender, MailParamsConstants.MAIL_SENDER + " must not null");
enableSmtpAuth = config.get(MailParamsConstants.NAME_MAIL_SMTP_AUTH);
mailUser = config.get(MailParamsConstants.NAME_MAIL_USER);
requireNonNull(mailUser, MailParamsConstants.MAIL_USER + " must not null");
mailPasswd = config.get(MailParamsConstants.NAME_MAIL_PASSWD);
requireNonNull(mailPasswd, MailParamsConstants.MAIL_PASSWD + " must not null");
mailUseStartTLS = config.get(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE);
requireNonNull(mailUseStartTLS, MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE + " must not null");
mailUseSSL = config.get(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE);
requireNonNull(mailUseSSL, MailParamsConstants.MAIL_SMTP_SSL_ENABLE + " must not null");
sslTrust = config.get(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST);
requireNonNull(sslTrust, MailParamsConstants.MAIL_SMTP_SSL_TRUST + " must not null");
showType = config.get(AlertConstants.SHOW_TYPE);
requireNonNull(showType, AlertConstants.SHOW_TYPE + " must not null");
xlsFilePath = config.get(EmailConstants.XLS_FILE_PATH);
if (StringUtils.isBlank(xlsFilePath)) {
xlsFilePath = "/tmp/xls";
}
alertTemplate = new DefaultHTMLTemplate();
}
/**
* send mail to receivers
*
* @param title title
* @param content content
* @return
*/
public AlertResult sendMails(String title, String content) {
return sendMails(this.receivers, this.receiverCcs, title, content);
}
/**
* send mail to receivers
*
* @param title email title
* @param content email content
* @return
*/
public AlertResult sendMailsToReceiverOnly(String title, String content) {
return sendMails(this.receivers, null, title, content);
}
/**
* send mail
*
* @param receivers receivers
* @param receiverCcs receiverCcs
* @param title title
* @param content content
* @return
*/
public AlertResult sendMails(List<String> receivers, List<String> receiverCcs, String title, String content) {
AlertResult alertResult = new AlertResult();
alertResult.setStatus("false");
// if there is no receivers && no receiversCc, no need to process
if (CollectionUtils.isEmpty(receivers) && CollectionUtils.isEmpty(receiverCcs)) {
return alertResult;
}
receivers.removeIf(StringUtils::isEmpty);
if (showType.equals(ShowType.TABLE.getDescp()) || showType.equals(ShowType.TEXT.getDescp())) {
// send email
HtmlEmail email = new HtmlEmail();
try {
Session session = getSession();
email.setMailSession(session);
email.setFrom(mailSender);
email.setCharset(EmailConstants.UTF_8);
if (CollectionUtils.isNotEmpty(receivers)) {
// receivers mail
for (String receiver : receivers) {
email.addTo(receiver);
}
}
if (CollectionUtils.isNotEmpty(receiverCcs)) {
//cc
for (String receiverCc : receiverCcs) {
email.addCc(receiverCc);
}
}
// sender mail
return getStringObjectMap(title, content, alertResult, email);
} catch (Exception e) {
handleException(alertResult, e);
}
} else if (showType.equals(ShowType.ATTACHMENT.getDescp()) || showType.equals(ShowType.TABLEATTACHMENT.getDescp())) {
try {
String partContent = (showType.equals(ShowType.ATTACHMENT.getDescp()) ? "Please see the attachment " + title + EmailConstants.EXCEL_SUFFIX_XLS : htmlTable(content, false));
attachment(title, content, partContent);
alertResult.setStatus("true");
return alertResult;
} catch (Exception e) {
handleException(alertResult, e);
return alertResult;
}
}
return alertResult;
}
/**
* html table content
*
* @param content the content
* @param showAll if show the whole content
* @return the html table form
*/
private String htmlTable(String content, boolean showAll) {
return alertTemplate.getMessageFromTemplate(content, ShowType.TABLE, showAll);
}
/**
* html table content
*
* @param content the content
* @return the html table form
*/
private String htmlTable(String content) {
return htmlTable(content, true);
}
/**
* html text content
*
* @param content the content
* @return text in html form
*/
private String htmlText(String content) {
return alertTemplate.getMessageFromTemplate(content, ShowType.TEXT);
}
/**
* send mail as Excel attachment
*
* @param title
* @param content
* @param partContent
* @throws Exception
*/
private void attachment(String title, String content, String partContent) throws Exception {
MimeMessage msg = getMimeMessage();
attachContent(title, content, partContent, msg);
}
/**
* get MimeMessage
*
* @return
* @throws MessagingException
*/
private MimeMessage getMimeMessage() throws MessagingException {
// 1. The first step in creating mail: creating session
Session session = getSession();
// Setting debug mode, can be turned off
session.setDebug(false);
// 2. creating mail: Creating a MimeMessage
MimeMessage msg = new MimeMessage(session);
// 3. set sender
msg.setFrom(new InternetAddress(mailSender));
// 4. set receivers
for (String receiver : receivers) {
msg.addRecipients(Message.RecipientType.TO, InternetAddress.parse(receiver));
}
return msg;
}
/**
* get session
*
* @return the new Session
*/
private Session getSession() {
Properties props = new Properties();
props.setProperty(MailParamsConstants.MAIL_SMTP_HOST, mailSmtpHost);
props.setProperty(MailParamsConstants.MAIL_SMTP_PORT, mailSmtpPort);
props.setProperty(MailParamsConstants.MAIL_SMTP_AUTH, enableSmtpAuth);
props.setProperty(EmailConstants.MAIL_TRANSPORT_PROTOCOL, mailProtocol);
props.setProperty(MailParamsConstants.MAIL_SMTP_STARTTLS_ENABLE, mailUseStartTLS);
props.setProperty(MailParamsConstants.MAIL_SMTP_SSL_ENABLE, mailUseSSL);
props.setProperty(MailParamsConstants.MAIL_SMTP_SSL_TRUST, sslTrust);
Authenticator auth = new Authenticator() {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
// mail username and password
return new PasswordAuthentication(mailUser, mailPasswd);
}
};
Session session = Session.getInstance(props, auth);
session.addProvider(new SMTPProvider());
return session;
}
/**
* attach content
*
* @param title
* @param content
* @param partContent
* @param msg
* @throws MessagingException
* @throws IOException
*/
private void attachContent(String title, String content, String partContent, MimeMessage msg) throws MessagingException, IOException {
/**
* set receiverCc
*/
if (CollectionUtils.isNotEmpty(receiverCcs)) {
for (String receiverCc : receiverCcs) {
msg.addRecipients(Message.RecipientType.CC, InternetAddress.parse(receiverCc));
}
}
// set subject
msg.setSubject(title);
MimeMultipart partList = new MimeMultipart();
// set signature
MimeBodyPart part1 = new MimeBodyPart();
part1.setContent(partContent, EmailConstants.TEXT_HTML_CHARSET_UTF_8);
// set attach file
MimeBodyPart part2 = new MimeBodyPart();
File file = new File(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS);
if (!file.getParentFile().exists()) {
file.getParentFile().mkdirs();
}
// make excel file
ExcelUtils.genExcelFile(content, title, xlsFilePath);
part2.attachFile(file);
part2.setFileName(MimeUtility.encodeText(title + EmailConstants.EXCEL_SUFFIX_XLS, EmailConstants.UTF_8, "B"));
// add components to collection
partList.addBodyPart(part1);
partList.addBodyPart(part2);
msg.setContent(partList);
// 5. send Transport
Transport.send(msg);
// 6. delete saved file
deleteFile(file);
}
/**
* the string object map
*
* @param title
* @param content
* @param alertResult
* @param email
* @return
* @throws EmailException
*/
private AlertResult getStringObjectMap(String title, String content, AlertResult alertResult, HtmlEmail email) throws EmailException {
/**
* the subject of the message to be sent
*/
email.setSubject(title);
/**
* to send information, you can use HTML tags in mail content because of the use of HtmlEmail
*/
if (showType.equals(ShowType.TABLE.getDescp())) {
email.setMsg(htmlTable(content));
} else if (showType.equals(ShowType.TEXT.getDescp())) {
email.setMsg(htmlText(content));
}
// send
email.setDebug(true);
Thread.currentThread().setContextClassLoader(getClass().getClassLoader());
email.send();
alertResult.setStatus("true");
return alertResult;
}
/**
* file delete
*
* @param file the file to delete
*/
public void deleteFile(File file) {
if (file.exists()) {
if (file.delete()) {
logger.info("delete success: {}", file.getAbsolutePath() + file.getName());
} else {
logger.info("delete fail: {}", file.getAbsolutePath() + file.getName());
}
} else {
logger.info("file not exists: {}", file.getAbsolutePath() + file.getName());
}
}
/**
* handle exception
*
* @param alertResult
* @param e
*/
private void handleException(AlertResult alertResult, Exception e) {
logger.error("Send email to {} failed", receivers, e);
alertResult.setMessage("Send email to {" + String.join(",", receivers) + "} failed," + e.toString());
}
}
......@@ -14,9 +14,10 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.template;
import org.apache.dolphinscheduler.common.enums.ShowType;
package org.apache.dolphinscheduler.plugin.alert.email.template;
import org.apache.dolphinscheduler.spi.alert.ShowType;
/**
* alert message template
......@@ -25,20 +26,22 @@ public interface AlertTemplate {
/**
* get a message from a specified alert template
* @param content alert message content
* @param showType show type
* @param showAll whether to show all
*
* @param content alert message content
* @param showType show type
* @param showAll whether to show all
* @return a message from a specified alert template
*/
String getMessageFromTemplate(String content, ShowType showType,boolean showAll);
String getMessageFromTemplate(String content, ShowType showType, boolean showAll);
/**
* default showAll is true
* @param content alert message content
*
* @param content alert message content
* @param showType show type
* @return a message from a specified alert template
*/
default String getMessageFromTemplate(String content,ShowType showType){
return getMessageFromTemplate(content,showType,true);
default String getMessageFromTemplate(String content, ShowType showType) {
return getMessageFromTemplate(content, showType, true);
}
}
......@@ -14,21 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.template.impl;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import org.apache.dolphinscheduler.alert.template.AlertTemplate;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.common.enums.ShowType;
import org.apache.dolphinscheduler.common.utils.StringUtils;
package org.apache.dolphinscheduler.plugin.alert.email.template;
import static java.util.Objects.requireNonNull;
import org.apache.dolphinscheduler.plugin.alert.email.EmailConstants;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.dolphinscheduler.common.utils.*;
import java.util.*;
import static org.apache.dolphinscheduler.common.utils.Preconditions.*;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
/**
* the default html alert message template
......@@ -37,33 +43,33 @@ public class DefaultHTMLTemplate implements AlertTemplate {
public static final Logger logger = LoggerFactory.getLogger(DefaultHTMLTemplate.class);
@Override
public String getMessageFromTemplate(String content, ShowType showType,boolean showAll) {
public String getMessageFromTemplate(String content, ShowType showType, boolean showAll) {
switch (showType){
switch (showType) {
case TABLE:
return getTableTypeMessage(content,showAll);
return getTableTypeMessage(content, showAll);
case TEXT:
return getTextTypeMessage(content,showAll);
return getTextTypeMessage(content, showAll);
default:
throw new IllegalArgumentException(String.format("not support showType: %s in DefaultHTMLTemplate",showType));
throw new IllegalArgumentException(String.format("not support showType: %s in DefaultHTMLTemplate", showType));
}
}
/**
* get alert message which type is TABLE
*
* @param content message content
* @param showAll weather to show all
* @return alert message
*/
private String getTableTypeMessage(String content,boolean showAll){
private String getTableTypeMessage(String content, boolean showAll) {
if (StringUtils.isNotEmpty(content)){
if (StringUtils.isNotEmpty(content)) {
List<LinkedHashMap> mapItemsList = JSONUtils.toList(content, LinkedHashMap.class);
if(!showAll && mapItemsList.size() > Constants.NUMBER_1000){
mapItemsList = mapItemsList.subList(0,Constants.NUMBER_1000);
if (!showAll && mapItemsList.size() > EmailConstants.NUMBER_1000) {
mapItemsList = mapItemsList.subList(0, EmailConstants.NUMBER_1000);
}
StringBuilder contents = new StringBuilder(200);
......@@ -71,31 +77,31 @@ public class DefaultHTMLTemplate implements AlertTemplate {
boolean flag = true;
String title = "";
for (LinkedHashMap mapItems : mapItemsList){
for (LinkedHashMap mapItems : mapItemsList) {
Set<Map.Entry<String, Object>> entries = mapItems.entrySet();
Iterator<Map.Entry<String, Object>> iterator = entries.iterator();
StringBuilder t = new StringBuilder(Constants.TR);
StringBuilder cs = new StringBuilder(Constants.TR);
while (iterator.hasNext()){
StringBuilder t = new StringBuilder(EmailConstants.TR);
StringBuilder cs = new StringBuilder(EmailConstants.TR);
while (iterator.hasNext()) {
Map.Entry<String, Object> entry = iterator.next();
t.append(Constants.TH).append(entry.getKey()).append(Constants.TH_END);
cs.append(Constants.TD).append(String.valueOf(entry.getValue())).append(Constants.TD_END);
t.append(EmailConstants.TH).append(entry.getKey()).append(EmailConstants.TH_END);
cs.append(EmailConstants.TD).append(String.valueOf(entry.getValue())).append(EmailConstants.TD_END);
}
t.append(Constants.TR_END);
cs.append(Constants.TR_END);
if (flag){
t.append(EmailConstants.TR_END);
cs.append(EmailConstants.TR_END);
if (flag) {
title = t.toString();
}
flag = false;
contents.append(cs);
}
return getMessageFromHtmlTemplate(title,contents.toString());
return getMessageFromHtmlTemplate(title, contents.toString());
}
return content;
......@@ -103,22 +109,23 @@ public class DefaultHTMLTemplate implements AlertTemplate {
/**
* get alert message which type is TEXT
*
* @param content message content
* @param showAll weather to show all
* @return alert message
*/
private String getTextTypeMessage(String content,boolean showAll){
private String getTextTypeMessage(String content, boolean showAll) {
if (StringUtils.isNotEmpty(content)){
if (StringUtils.isNotEmpty(content)) {
ArrayNode list = JSONUtils.parseArray(content);
StringBuilder contents = new StringBuilder(100);
for (JsonNode jsonNode : list){
contents.append(Constants.TR);
contents.append(Constants.TD).append(jsonNode.toString()).append(Constants.TD_END);
contents.append(Constants.TR_END);
for (JsonNode jsonNode : list) {
contents.append(EmailConstants.TR);
contents.append(EmailConstants.TD).append(jsonNode.toString()).append(EmailConstants.TD_END);
contents.append(EmailConstants.TR_END);
}
return getMessageFromHtmlTemplate(null,contents.toString());
return getMessageFromHtmlTemplate(null, contents.toString());
}
......@@ -127,16 +134,17 @@ public class DefaultHTMLTemplate implements AlertTemplate {
/**
* get alert message from a html template
* @param title message title
* @param content message content
*
* @param title message title
* @param content message content
* @return alert message which use html template
*/
private String getMessageFromHtmlTemplate(String title,String content){
private String getMessageFromHtmlTemplate(String title, String content) {
checkNotNull(content);
String htmlTableThead = StringUtils.isEmpty(title) ? "" : String.format("<thead>%s</thead>\n",title);
requireNonNull(content, "content must not null");
String htmlTableThead = StringUtils.isEmpty(title) ? "" : String.format("<thead>%s</thead>\n", title);
return Constants.HTML_HEADER_PREFIX +htmlTableThead + content + Constants.TABLE_BODY_HTML_TAIL;
return EmailConstants.HTML_HEADER_PREFIX + htmlTableThead + content + EmailConstants.TABLE_BODY_HTML_TAIL;
}
}
......@@ -14,67 +14,62 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.model;
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;
package org.apache.dolphinscheduler.plugin.alert.email;
public class AlertDataTest {
import org.apache.dolphinscheduler.spi.alert.AlertChannel;
import org.apache.dolphinscheduler.spi.params.base.PluginParams;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
private AlertData alertData;
@Before
public void before() {
alertData = new AlertData();
alertData.setId(1)
.setContent("content")
.setShowType("email")
.setTitle("title")
.setReceivers("receivers")
.setReceiversCc("cc")
.setLog("log")
.setAlertGroupId(1);
}
import java.util.List;
@Test
public void getId() {
assertEquals(1, alertData.getId());
}
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
@Test
public void getTitle() {
assertEquals("title", alertData.getTitle());
}
/**
* EmailAlertChannelFactory Tester.
*
* @version 1.0
* @since <pre>Aug 20, 2020</pre>
*/
public class EmailAlertChannelFactoryTest {
@Test
public void getContent() {
assertEquals("content", alertData.getContent());
@Before
public void before() throws Exception {
}
@Test
public void getLog() {
assertEquals("log", alertData.getLog());
@After
public void after() throws Exception {
}
/**
* Method: getName()
*/
@Test
public void getAlertGroupId() {
assertEquals(1, alertData.getAlertGroupId());
public void testGetName() throws Exception {
}
/**
* Method: getParams()
*/
@Test
public void getReceivers() {
assertEquals("receivers", alertData.getReceivers());
public void testGetParams() throws Exception {
EmailAlertChannelFactory emailAlertChannelFactory = new EmailAlertChannelFactory();
List<PluginParams> params = emailAlertChannelFactory.getParams();
JSONUtils.toJsonString(params);
Assert.assertEquals(12, params.size());
}
/**
* Method: create()
*/
@Test
public void getReceiversCc() {
assertEquals("cc", alertData.getReceiversCc());
public void testCreate() throws Exception {
EmailAlertChannelFactory emailAlertChannelFactory = new EmailAlertChannelFactory();
AlertChannel alertChannel = emailAlertChannelFactory.create();
Assert.assertNotNull(alertChannel);
}
@Test
public void getShowType() {
assertEquals("email", alertData.getShowType());
}
}
\ No newline at end of file
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
import org.apache.dolphinscheduler.spi.alert.AlertConstants;
import org.apache.dolphinscheduler.spi.alert.AlertData;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import org.apache.dolphinscheduler.spi.alert.AlertResult;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.params.InputParam;
import org.apache.dolphinscheduler.spi.params.PasswordParam;
import org.apache.dolphinscheduler.spi.params.RadioParam;
import org.apache.dolphinscheduler.spi.params.base.DataType;
import org.apache.dolphinscheduler.spi.params.base.ParamsOptions;
import org.apache.dolphinscheduler.spi.params.base.PluginParams;
import org.apache.dolphinscheduler.spi.params.base.Validate;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
/**
* EmailAlertChannel Tester.
*
* @version 1.0
* @since <pre>Aug 20, 2020</pre>
*/
public class EmailAlertChannelTest {
@Before
public void before() throws Exception {
}
@After
public void after() throws Exception {
}
/**
* Method: process(AlertInfo info)
*/
@Test
public void testProcess() throws Exception {
EmailAlertChannel emailAlertChannel = new EmailAlertChannel();
AlertData alertData = new AlertData();
LinkedHashMap<String, Object> map1 = new LinkedHashMap<>();
map1.put("mysql service name", "mysql200");
map1.put("mysql address", "192.168.xx.xx");
map1.put("port", "3306");
map1.put("no index of number", "80");
map1.put("database client connections", "190");
List<LinkedHashMap<String, Object>> maps = new ArrayList<>();
maps.add(0, map1);
String mapjson = JSONUtils.toJsonString(maps);
alertData.setId(10)
.setContent(mapjson)
.setLog("10")
.setTitle("test");
AlertInfo alertInfo = new AlertInfo();
alertInfo.setAlertData(alertData);
alertInfo.setAlertParams(getEmailAlertParams());
AlertResult alertResult = emailAlertChannel.process(alertInfo);
Assert.assertNotNull(alertResult);
Assert.assertEquals("false", alertResult.getStatus());
}
public String getEmailAlertParams() {
List<PluginParams> paramsList = new ArrayList<>();
InputParam receivesParam = InputParam.newBuilder("receivers", "receivers")
.setValue("540957506@qq.com")
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
InputParam mailSmtpHost = InputParam.newBuilder("mailServerHost", "mail.smtp.host")
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue("smtp.126.com")
.build();
InputParam mailSmtpPort = InputParam.newBuilder("mailServerPort", "mail.smtp.port")
.addValidate(Validate.newBuilder()
.setRequired(true)
.setType(DataType.NUMBER.getDataType())
.build())
.setValue(25)
.build();
InputParam mailSender = InputParam.newBuilder("mailSender", "mail.sender")
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue("dolphinscheduler@126.com")
.build();
RadioParam enableSmtpAuth = RadioParam.newBuilder("enableSmtpAuth", "mail.smtp.auth")
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue(false)
.build();
InputParam mailUser = InputParam.newBuilder("mailUser", "mail.user")
.setPlaceholder("if enable use authentication, you need input user")
.setValue("dolphinscheduler@126.com")
.build();
PasswordParam mailPassword = PasswordParam.newBuilder("mailPasswd", "mail.passwd")
.setPlaceholder("if enable use authentication, you need input password")
.setValue("escheduler123")
.build();
RadioParam enableTls = RadioParam.newBuilder("starttlsEnable", "mail.smtp.starttls.enable")
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue(true)
.build();
RadioParam enableSsl = RadioParam.newBuilder("sslEnable", "mail.smtp.ssl.enable")
.addParamsOptions(new ParamsOptions("YES", true, false))
.addParamsOptions(new ParamsOptions("NO", false, false))
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue(true)
.build();
InputParam sslTrust = InputParam.newBuilder("mailSmtpSslTrust", "mail.smtp.ssl.trust")
.addValidate(Validate.newBuilder().setRequired(true).build())
.setValue("smtp.126.com")
.build();
List<ParamsOptions> emailShowTypeList = new ArrayList<>();
emailShowTypeList.add(new ParamsOptions(ShowType.TABLE.getDescp(), ShowType.TABLE.getDescp(), false));
emailShowTypeList.add(new ParamsOptions(ShowType.TEXT.getDescp(), ShowType.TEXT.getDescp(), false));
emailShowTypeList.add(new ParamsOptions(ShowType.ATTACHMENT.getDescp(), ShowType.ATTACHMENT.getDescp(), false));
emailShowTypeList.add(new ParamsOptions(ShowType.TABLEATTACHMENT.getDescp(), ShowType.TABLEATTACHMENT.getDescp(), false));
RadioParam showType = RadioParam.newBuilder(AlertConstants.SHOW_TYPE, "showType")
.setParamsOptionsList(emailShowTypeList)
.setValue(ShowType.TABLE.getDescp())
.addValidate(Validate.newBuilder().setRequired(true).build())
.build();
paramsList.add(receivesParam);
paramsList.add(mailSmtpHost);
paramsList.add(mailSmtpPort);
paramsList.add(mailSender);
paramsList.add(enableSmtpAuth);
paramsList.add(mailUser);
paramsList.add(mailPassword);
paramsList.add(enableTls);
paramsList.add(enableSsl);
paramsList.add(sslTrust);
paramsList.add(showType);
return JSONUtils.toJsonString(paramsList);
}
}
......@@ -15,7 +15,11 @@
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
package org.apache.dolphinscheduler.plugin.alert.email;
import static org.junit.Assert.assertTrue;
import java.io.File;
import org.junit.After;
import org.junit.Before;
......@@ -25,8 +29,6 @@ import org.junit.rules.ExpectedException;
import org.junit.rules.TemporaryFolder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import static org.junit.Assert.assertTrue;
public class ExcelUtilsTest {
......@@ -61,7 +63,7 @@ public class ExcelUtilsTest {
//Define dest file path
String xlsFilePath = rootPath + System.getProperty("file.separator");
logger.info("xlsFilePath: "+xlsFilePath);
logger.info("xlsFilePath: " + xlsFilePath);
//Define correctContent
String correctContent = "[{\"name\":\"ds name\",\"value\":\"ds value\"}]";
......@@ -76,7 +78,7 @@ public class ExcelUtilsTest {
ExcelUtils.genExcelFile(correctContent, title, xlsFilePath);
//Test file exists
File xlsFile = new File(xlsFilePath + Constants.SINGLE_SLASH + title + Constants.EXCEL_SUFFIX_XLS);
File xlsFile = new File(xlsFilePath + EmailConstants.SINGLE_SLASH + title + EmailConstants.EXCEL_SUFFIX_XLS);
assertTrue(xlsFile.exists());
//Expected RuntimeException
......@@ -96,7 +98,7 @@ public class ExcelUtilsTest {
@Test
public void testGenExcelFileByCheckDir() {
ExcelUtils.genExcelFile("[{\"a\": \"a\"},{\"a\": \"a\"}]", "t", "/tmp/xls");
File file = new File("/tmp/xls" + Constants.SINGLE_SLASH + "t" + Constants.EXCEL_SUFFIX_XLS);
File file = new File("/tmp/xls" + EmailConstants.SINGLE_SLASH + "t" + EmailConstants.EXCEL_SUFFIX_XLS);
file.delete();
}
}
\ No newline at end of file
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.alert.email;
import org.apache.dolphinscheduler.plugin.alert.email.template.AlertTemplate;
import org.apache.dolphinscheduler.plugin.alert.email.template.DefaultHTMLTemplate;
import org.apache.dolphinscheduler.spi.alert.AlertConstants;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.junit.BeforeClass;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
*/
public class MailUtilsTest {
private static final Logger logger = LoggerFactory.getLogger(MailUtilsTest.class);
private static Map<String, String> emailConfig = new HashMap<>();
private static AlertTemplate alertTemplate;
static MailSender mailSender;
@BeforeClass
public static void initEmailConfig() {
emailConfig.put(MailParamsConstants.NAME_MAIL_PROTOCOL, "smtp");
emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_HOST, "xxx.xxx.com");
emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_PORT, "25");
emailConfig.put(MailParamsConstants.NAME_MAIL_SENDER, "xxx1.xxx.com");
emailConfig.put(MailParamsConstants.NAME_MAIL_USER, "xxx2.xxx.com");
emailConfig.put(MailParamsConstants.NAME_MAIL_PASSWD, "111111");
emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_STARTTLS_ENABLE, "true");
emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_SSL_ENABLE, "false");
emailConfig.put(MailParamsConstants.NAME_MAIL_SMTP_SSL_TRUST, "false");
emailConfig.put(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERS, "347801120@qq.com");
emailConfig.put(MailParamsConstants.NAME_PLUGIN_DEFAULT_EMAIL_RECEIVERCCS, "347801120@qq.com");
emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TEXT.getDescp());
alertTemplate = new DefaultHTMLTemplate();
mailSender = new MailSender(emailConfig);
}
@Test
public void testSendMails() {
String content = "[\"id:69\","
+ "\"name:UserBehavior-0--1193959466\","
+ "\"Job name: Start workflow\","
+ "\"State: SUCCESS\","
+ "\"Recovery:NO\","
+ "\"Run time: 1\","
+ "\"Start time: 2018-08-06 10:31:34.0\","
+ "\"End time: 2018-08-06 10:31:49.0\","
+ "\"Host: 192.168.xx.xx\","
+ "\"Notify group :4\"]";
mailSender.sendMails(
"Mysql Exception",
content);
}
public String list2String() {
LinkedHashMap<String, Object> map1 = new LinkedHashMap<>();
map1.put("mysql service name", "mysql200");
map1.put("mysql address", "192.168.xx.xx");
map1.put("port", "3306");
map1.put("no index of number", "80");
map1.put("database client connections", "190");
LinkedHashMap<String, Object> map2 = new LinkedHashMap<>();
map2.put("mysql service name", "mysql210");
map2.put("mysql address", "192.168.xx.xx");
map2.put("port", "3306");
map2.put("no index of number", "10");
map2.put("database client connections", "90");
List<LinkedHashMap<String, Object>> maps = new ArrayList<>();
maps.add(0, map1);
maps.add(1, map2);
String mapjson = JSONUtils.toJsonString(maps);
logger.info(mapjson);
return mapjson;
}
@Test
public void testSendTableMail() {
String title = "Mysql Exception";
String content = list2String();
emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TABLE.getDescp());
mailSender = new MailSender(emailConfig);
mailSender.sendMails(title, content);
}
@Test
public void testAttachmentFile() throws Exception {
String content = list2String();
emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.ATTACHMENT.getDescp());
mailSender = new MailSender(emailConfig);
mailSender.sendMails("gaojing", content);
}
@Test
public void testTableAttachmentFile() throws Exception {
String content = list2String();
emailConfig.put(AlertConstants.SHOW_TYPE, ShowType.TABLEATTACHMENT.getDescp());
mailSender = new MailSender(emailConfig);
mailSender.sendMails("gaojing", content);
}
}
......@@ -14,25 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.template.impl;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.common.enums.ShowType;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
package org.apache.dolphinscheduler.plugin.alert.email.template;
import static org.junit.Assert.assertEquals;
import org.apache.dolphinscheduler.plugin.alert.email.EmailConstants;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.utils.JSONUtils;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import static org.junit.Assert.*;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* test class for DefaultHTMLTemplate
*/
public class DefaultHTMLTemplateTest{
public class DefaultHTMLTemplateTest {
private static final Logger logger = LoggerFactory.getLogger(DefaultHTMLTemplateTest.class);
......@@ -40,57 +42,64 @@ public class DefaultHTMLTemplateTest{
* only need test method GetMessageFromTemplate
*/
@Test
public void testGetMessageFromTemplate(){
public void testGetMessageFromTemplate() {
DefaultHTMLTemplate template = new DefaultHTMLTemplate();
String tableTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TABLE,true);
String tableTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TABLE, true);
assertEquals(tableTypeMessage,generateMockTableTypeResultByHand());
assertEquals(tableTypeMessage, generateMockTableTypeResultByHand());
String textTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TEXT,true);
String textTypeMessage = template.getMessageFromTemplate(list2String(), ShowType.TEXT, true);
assertEquals(textTypeMessage,generateMockTextTypeResultByHand());
assertEquals(textTypeMessage, generateMockTextTypeResultByHand());
}
/**
* generate some simulation data
*/
private String list2String(){
private String list2String() {
LinkedHashMap<String, Object> map1 = new LinkedHashMap<>();
map1.put("mysql service name","mysql200");
map1.put("mysql address","192.168.xx.xx");
map1.put("database client connections","190");
map1.put("port","3306");
map1.put("no index of number","80");
map1.put("mysql service name", "mysql200");
map1.put("mysql address", "192.168.xx.xx");
map1.put("database client connections", "190");
map1.put("port", "3306");
map1.put("no index of number", "80");
LinkedHashMap<String, Object> map2 = new LinkedHashMap<>();
map2.put("mysql service name","mysql210");
map2.put("mysql address","192.168.xx.xx");
map2.put("database client connections","90");
map2.put("port","3306");
map2.put("no index of number","10");
map2.put("mysql service name", "mysql210");
map2.put("mysql address", "192.168.xx.xx");
map2.put("database client connections", "90");
map2.put("port", "3306");
map2.put("no index of number", "10");
List<LinkedHashMap<String, Object>> maps = new ArrayList<>();
maps.add(0,map1);
maps.add(1,map2);
maps.add(0, map1);
maps.add(1, map2);
String mapjson = JSONUtils.toJsonString(maps);
logger.info(mapjson);
return mapjson;
}
private String generateMockTableTypeResultByHand(){
private String generateMockTableTypeResultByHand() {
return Constants.HTML_HEADER_PREFIX +
"<thead><tr><th>mysql service name</th><th>mysql address</th><th>database client connections</th><th>port</th><th>no index of number</th></tr></thead>\n" +
"<tr><td>mysql200</td><td>192.168.xx.xx</td><td>190</td><td>3306</td><td>80</td></tr><tr><td>mysql210</td><td>192.168.xx.xx</td><td>90</td><td>3306</td><td>10</td></tr>" + Constants.TABLE_BODY_HTML_TAIL;
return EmailConstants.HTML_HEADER_PREFIX
+ "<thead>"
+ "<tr><th>mysql service name</th><th>mysql address</th><th>database client connections</th><th>port</th><th>no index of number</th></tr>"
+ "</thead>\n"
+ "<tr><td>mysql200</td><td>192.168.xx.xx</td><td>190</td><td>3306</td><td>80</td></tr>"
+ "<tr><td>mysql210</td><td>192.168.xx.xx</td><td>90</td><td>3306</td><td>10</td></tr>"
+ EmailConstants.TABLE_BODY_HTML_TAIL;
}
private String generateMockTextTypeResultByHand(){
private String generateMockTextTypeResultByHand() {
return Constants.HTML_HEADER_PREFIX + "<tr><td>{\"mysql service name\":\"mysql200\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"190\",\"port\":\"3306\",\"no index of number\":\"80\"}</td></tr><tr><td>{\"mysql service name\":\"mysql210\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"90\",\"port\":\"3306\",\"no index of number\":\"10\"}</td></tr>" + Constants.TABLE_BODY_HTML_TAIL;
return EmailConstants.HTML_HEADER_PREFIX
+ "<tr><td>{\"mysql service name\":\"mysql200\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"190\",\"port\":\"3306\",\"no index of number\":\"80\"}</td></tr>"
+ "<tr><td>{\"mysql service name\":\"mysql210\",\"mysql address\":\"192.168.xx.xx\",\"database client connections\":\"90\",\"port\":\"3306\",\"no index of number\":\"10\"}</td></tr>"
+ EmailConstants.TABLE_BODY_HTML_TAIL;
}
}
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert-wechat</artifactId>
</project>
\ No newline at end of file
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>dolphinscheduler</artifactId>
<groupId>org.apache.dolphinscheduler</groupId>
<version>1.3.2-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-alert-plugin</artifactId>
<packaging>pom</packaging>
<modules>
<module>dolphinscheduler-alert-email</module>
<module>dolphinscheduler-alert-wechat</module>
<module>dolphinscheduler-alert-dingtalk</module>
</modules>
</project>
\ No newline at end of file
......@@ -32,38 +32,13 @@
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<type>jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<scope>test</scope>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-email</artifactId>
</dependency>
<dependency>
......@@ -108,6 +83,53 @@
</exclusions>
</dependency>
<dependency>
<groupId>org.sonatype.aether</groupId>
<artifactId>aether-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.airlift.resolver</groupId>
<artifactId>resolver</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-module-junit4</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-api-mockito2</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
......@@ -14,21 +14,28 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert;
import org.apache.dolphinscheduler.alert.plugin.EmailAlertPlugin;
import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager;
import org.apache.dolphinscheduler.alert.plugin.DolphinPluginLoader;
import org.apache.dolphinscheduler.alert.plugin.DolphinPluginManagerConfig;
import org.apache.dolphinscheduler.alert.runner.AlertSender;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.utils.PropertyUtils;
import org.apache.dolphinscheduler.common.plugin.FilePluginManager;
import org.apache.dolphinscheduler.common.thread.Stopper;
import org.apache.dolphinscheduler.dao.AlertDao;
import org.apache.dolphinscheduler.dao.DaoFactory;
import org.apache.dolphinscheduler.dao.PluginDao;
import org.apache.dolphinscheduler.dao.entity.Alert;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.List;
import com.google.common.collect.ImmutableList;
/**
* alert of start
......@@ -40,35 +47,59 @@ public class AlertServer {
*/
private AlertDao alertDao = DaoFactory.getDaoInstance(AlertDao.class);
private PluginDao pluginDao = DaoFactory.getDaoInstance(PluginDao.class);
private AlertSender alertSender;
private static AlertServer instance;
private FilePluginManager alertPluginManager;
private AlertPluginManager alertPluginManager;
private DolphinPluginManagerConfig alertPluginManagerConfig;
public static final String ALERT_PLUGIN_BINDING = "alert.plugin.binding";
public static final String ALERT_PLUGIN_DIR = "alert.plugin.dir";
public static final String MAVEN_LOCAL_REPOSITORY = "maven.local.repository";
private static class AlertServerHolder {
private static final AlertServer INSTANCE = new AlertServer();
}
public static final AlertServer getInstance() {
return AlertServerHolder.INSTANCE;
private static final String[] whitePrefixes = new String[]{"org.apache.dolphinscheduler.plugin.utils."};
}
private static final String[] excludePrefixes = new String[]{
"org.apache.dolphinscheduler.plugin.",
"ch.qos.logback.",
"org.slf4j."
};
private AlertServer() {
public AlertServer() {
alertPluginManager =
new FilePluginManager(PropertyUtils.getString(Constants.PLUGIN_DIR), whitePrefixes, excludePrefixes);
// add default alert plugins
alertPluginManager.addPlugin(new EmailAlertPlugin());
}
public synchronized static AlertServer getInstance() {
if (null == instance) {
instance = new AlertServer();
private void initPlugin() {
alertPluginManager = new AlertPluginManager();
alertPluginManagerConfig = new DolphinPluginManagerConfig();
alertPluginManagerConfig.setPlugins(PropertyUtils.getString(ALERT_PLUGIN_BINDING));
if (StringUtils.isNotBlank(PropertyUtils.getString(ALERT_PLUGIN_DIR))) {
alertPluginManagerConfig.setInstalledPluginsDir(PropertyUtils.getString(ALERT_PLUGIN_DIR, Constants.ALERT_PLUGIN_PATH).trim());
}
if (StringUtils.isNotBlank(PropertyUtils.getString(MAVEN_LOCAL_REPOSITORY))) {
alertPluginManagerConfig.setMavenLocalRepository(PropertyUtils.getString(MAVEN_LOCAL_REPOSITORY).trim());
}
DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager));
try {
alertPluginLoader.loadPlugins();
} catch (Exception e) {
throw new RuntimeException("load Alert Plugin Failed !", e);
}
return instance;
}
public void start() {
initPlugin();
logger.info("alert server ready start ");
while (Stopper.isRunning()) {
try {
......@@ -77,14 +108,18 @@ public class AlertServer {
logger.error(e.getMessage(), e);
Thread.currentThread().interrupt();
}
List<Alert> alerts = alertDao.listWaitExecutionAlert();
alertSender = new AlertSender(alerts, alertDao, alertPluginManager);
alertSender.run();
if (alertPluginManager == null || alertPluginManager.getAlertChannelMap().size() == 0) {
logger.warn("No Alert Plugin . Can not send alert info. ");
} else {
List<Alert> alerts = alertDao.listWaitExecutionAlert();
alertSender = new AlertSender(alerts, alertDao, alertPluginManager, pluginDao);
alertSender.run();
}
}
}
public static void main(String[] args) {
System.out.println(System.getProperty("user.dir"));
AlertServer alertServer = AlertServer.getInstance();
alertServer.start();
}
......
......@@ -14,33 +14,35 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.manager;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.utils.DingTalkUtils;
import org.apache.dolphinscheduler.plugin.model.AlertInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Ding Talk Manager
*/
public class DingTalkManager {
private static final Logger logger = LoggerFactory.getLogger(EnterpriseWeChatManager.class);
public Map<String,Object> send(AlertInfo alert) {
Map<String,Object> retMap = new HashMap<>();
public Map<String, Object> send(AlertInfo alert) {
Map<String, Object> retMap = new HashMap<>();
retMap.put(Constants.STATUS, false);
logger.info("send message {}", alert.getAlertData().getTitle());
try {
String msg = buildMessage(alert);
DingTalkUtils.sendDingTalkMsg(msg, Constants.UTF_8);
} catch (IOException e) {
logger.error(e.getMessage(),e);
logger.error(e.getMessage(), e);
}
retMap.put(Constants.STATUS, true);
return retMap;
......
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.manager;
import org.apache.dolphinscheduler.alert.utils.MailUtils;
import java.util.List;
import java.util.Map;
/**
* email send manager
*/
public class EmailManager {
/**
* email send
* @param receiversList the receiver list
* @param receiversCcList the cc List
* @param title the title
* @param content the content
* @param showType the showType
* @return the send result
*/
public Map<String,Object> send(List<String> receiversList,List<String> receiversCcList,String title,String content,String showType){
return MailUtils.sendMails(receiversList, receiversCcList, title, content, showType);
}
/**
* msg send
* @param receiversList the receiver list
* @param title the title
* @param content the content
* @param showType the showType
* @return the send result
*/
public Map<String,Object> send(List<String> receiversList,String title,String content,String showType){
return MailUtils.sendMails(receiversList,title, content, showType);
}
}
///*
// * Licensed to the Apache Software Foundation (ASF) under one or more
// * contributor license agreements. See the NOTICE file distributed with
// * this work for additional information regarding copyright ownership.
// * The ASF licenses this file to You under the Apache License, Version 2.0
// * (the "License"); you may not use this file except in compliance with
// * the License. You may obtain a copy of the License at
// *
// * http://www.apache.org/licenses/LICENSE-2.0
// *
// * Unless required by applicable law or agreed to in writing, software
// * distributed under the License is distributed on an "AS IS" BASIS,
// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// * See the License for the specific language governing permissions and
// * limitations under the License.
// */
//package org.apache.dolphinscheduler.alert.manager;
//
//import org.apache.dolphinscheduler.alert.utils.MailUtils;
//
//import java.util.List;
//import java.util.Map;
//
///**
// * email send manager
// */
//public class EmailManager {
// /**
// * email send
// * @param receiversList the receiver list
// * @param receiversCcList the cc List
// * @param title the title
// * @param content the content
// * @param showType the showType
// * @return the send result
// */
// public Map<String,Object> send(List<String> receiversList,List<String> receiversCcList,String title,String content,String showType){
//
// return MailUtils.sendMails(receiversList, receiversCcList, title, content, showType);
// }
//
// /**
// * msg send
// * @param receiversList the receiver list
// * @param title the title
// * @param content the content
// * @param showType the showType
// * @return the send result
// */
// public Map<String,Object> send(List<String> receiversList,String title,String content,String showType){
//
// return MailUtils.sendMails(receiversList,title, content, showType);
// }
//
//}
......@@ -14,13 +14,12 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.manager;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.utils.EnterpriseWeChatUtils;
import org.apache.dolphinscheduler.plugin.model.AlertInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import java.io.IOException;
import java.util.Arrays;
......@@ -28,29 +27,34 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Enterprise WeChat Manager
*/
public class EnterpriseWeChatManager {
private static final Logger logger = LoggerFactory.getLogger(EnterpriseWeChatManager.class);
/**
* Enterprise We Chat send
*
* @param alertInfo the alert info
* @param token the token
* @param token the token
* @return the send result
*/
public Map<String,Object> send(AlertInfo alertInfo, String token){
Map<String,Object> retMap = new HashMap<>();
public Map<String, Object> send(AlertInfo alertInfo, String token) {
Map<String, Object> retMap = new HashMap<>();
retMap.put(Constants.STATUS, false);
String agentId = EnterpriseWeChatUtils.ENTERPRISE_WE_CHAT_AGENT_ID;
String users = EnterpriseWeChatUtils.ENTERPRISE_WE_CHAT_USERS;
List<String> userList = Arrays.asList(users.split(","));
logger.info("send message {}", alertInfo.getAlertData().getTitle());
String msg = EnterpriseWeChatUtils.makeUserSendMsg(userList, agentId,EnterpriseWeChatUtils.markdownByAlert(alertInfo.getAlertData()));
String msg = EnterpriseWeChatUtils.makeUserSendMsg(userList, agentId, EnterpriseWeChatUtils.markdownByAlert(alertInfo));
try {
EnterpriseWeChatUtils.sendEnterpriseWeChat(Constants.UTF_8, msg, token);
} catch (IOException e) {
logger.error(e.getMessage(),e);
logger.error(e.getMessage(), e);
}
retMap.put(Constants.STATUS, true);
return retMap;
......
......@@ -14,23 +14,27 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.manager;
import org.apache.dolphinscheduler.dao.entity.Alert;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* SMS send manager
*/
public class MsgManager {
public class MsgManager {
private static final Logger logger = LoggerFactory.getLogger(MsgManager.class);
/**
* SMS send
*
* @param alert the alert
*/
public void send(Alert alert){
logger.info("send message {}",alert);
public void send(Alert alert) {
logger.info("send message {}", alert);
}
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import org.apache.dolphinscheduler.dao.DaoFactory;
import org.apache.dolphinscheduler.dao.PluginDao;
import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin;
public abstract class AbstractDolphinPluginManager {
protected PluginDao pluginDao = DaoFactory.getDaoInstance(PluginDao.class);
public abstract void installPlugin(DolphinSchedulerPlugin dolphinSchedulerPlugin);
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import static java.lang.String.format;
import static java.util.Objects.requireNonNull;
import static com.google.common.base.Preconditions.checkState;
import org.apache.dolphinscheduler.dao.entity.PluginDefine;
import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin;
import org.apache.dolphinscheduler.spi.alert.AlertChannel;
import org.apache.dolphinscheduler.spi.alert.AlertChannelFactory;
import org.apache.dolphinscheduler.spi.classloader.ThreadContextClassLoader;
import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer;
import org.apache.dolphinscheduler.spi.params.base.PluginParams;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* load the configured alert plugin and manager them
*/
public class AlertPluginManager extends AbstractDolphinPluginManager {
private static final Logger logger = LoggerFactory.getLogger(AlertPluginManager.class);
private final Map<String, AlertChannelFactory> alertChannelFactoryMap = new ConcurrentHashMap<>();
private final Map<String, AlertChannel> alertChannelMap = new ConcurrentHashMap<>();
public void addAlertChannelFactory(AlertChannelFactory alertChannelFactory) {
requireNonNull(alertChannelFactory, "alertChannelFactory is null");
if (alertChannelFactoryMap.putIfAbsent(alertChannelFactory.getName(), alertChannelFactory) != null) {
throw new IllegalArgumentException(format("Alert Plugin '{}' is already registered", alertChannelFactory.getName()));
}
try {
loadAlertChannel(alertChannelFactory.getName());
} catch (Exception e) {
throw new IllegalArgumentException(format("Alert Plugin '{}' is can not load .", alertChannelFactory.getName()));
}
}
protected void loadAlertChannel(String name) {
requireNonNull(name, "name is null");
AlertChannelFactory alertChannelFactory = alertChannelFactoryMap.get(name);
checkState(alertChannelFactory != null, "Alert Plugin {} is not registered", name);
try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(alertChannelFactory.getClass().getClassLoader())) {
AlertChannel alertChannel = alertChannelFactory.create();
this.alertChannelMap.put(name, alertChannel);
}
logger.info("-- Loaded Alert Plugin {} --", name);
}
public Map<String, AlertChannelFactory> getAlertChannelFactoryMap() {
return alertChannelFactoryMap;
}
public Map<String, AlertChannel> getAlertChannelMap() {
return alertChannelMap;
}
@Override
public void installPlugin(DolphinSchedulerPlugin dolphinSchedulerPlugin) {
for (AlertChannelFactory alertChannelFactory : dolphinSchedulerPlugin.getAlertChannelFactorys()) {
logger.info("Registering Alert Plugin '{}'", alertChannelFactory.getName());
this.addAlertChannelFactory(alertChannelFactory);
List<PluginParams> params = alertChannelFactory.getParams();
String nameEn = alertChannelFactory.getName();
String paramsJson = PluginParamsTransfer.transferParamsToJson(params);
PluginDefine pluginDefine = new PluginDefine(nameEn, "alert", paramsJson);
pluginDao.addOrUpdatePluginDefine(pluginDefine);
}
}
}
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import static java.util.Objects.requireNonNull;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Enumeration;
import java.util.List;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;
class DolphinPluginClassLoader
extends URLClassLoader {
private static final ClassLoader PLATFORM_CLASS_LOADER = findPlatformClassLoader();
private final ClassLoader spiClassLoader;
private final List<String> spiPackages;
private final List<String> spiResources;
public DolphinPluginClassLoader(
List<URL> urls,
ClassLoader spiClassLoader,
Iterable<String> spiPackages) {
this(urls,
spiClassLoader,
spiPackages,
Iterables.transform(spiPackages, DolphinPluginClassLoader::classNameToResource));
}
private DolphinPluginClassLoader(
List<URL> urls,
ClassLoader spiClassLoader,
Iterable<String> spiPackages,
Iterable<String> spiResources) {
// plugins should not have access to the system (application) class loader
super(urls.toArray(new URL[urls.size()]), PLATFORM_CLASS_LOADER);
this.spiClassLoader = requireNonNull(spiClassLoader, "spiClassLoader is null");
this.spiPackages = ImmutableList.copyOf(spiPackages);
this.spiResources = ImmutableList.copyOf(spiResources);
}
@Override
protected Class<?> loadClass(String name, boolean resolve)
throws ClassNotFoundException {
// grab the magic lock
synchronized (getClassLoadingLock(name)) {
// Check if class is in the loaded classes cache
Class<?> cachedClass = findLoadedClass(name);
if (cachedClass != null) {
return resolveClass(cachedClass, resolve);
}
// If this is an SPI class, only check SPI class loader
if (isSpiClass(name)) {
return resolveClass(spiClassLoader.loadClass(name), resolve);
}
// Look for class locally
return super.loadClass(name, resolve);
}
}
private Class<?> resolveClass(Class<?> clazz, boolean resolve) {
if (resolve) {
resolveClass(clazz);
}
return clazz;
}
@Override
public URL getResource(String name) {
// If this is an SPI resource, only check SPI class loader
if (isSpiResource(name)) {
return spiClassLoader.getResource(name);
}
// Look for resource locally
return super.getResource(name);
}
@Override
public Enumeration<URL> getResources(String name)
throws IOException {
// If this is an SPI resource, use SPI resources
if (isSpiClass(name)) {
return spiClassLoader.getResources(name);
}
// Use local resources
return super.getResources(name);
}
private boolean isSpiClass(String name) {
return spiPackages.stream().anyMatch(name::startsWith);
}
private boolean isSpiResource(String name) {
return spiResources.stream().anyMatch(name::startsWith);
}
private static String classNameToResource(String className) {
return className.replace('.', '/');
}
@SuppressWarnings("JavaReflectionMemberAccess")
private static ClassLoader findPlatformClassLoader() {
try {
// use platform class loader on Java 9
Method method = ClassLoader.class.getMethod("getPlatformClassLoader");
return (ClassLoader) method.invoke(null);
} catch (NoSuchMethodException ignored) {
// use null class loader on Java 8
return null;
} catch (IllegalAccessException | InvocationTargetException e) {
throw new AssertionError(e);
}
}
}
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.nio.file.Files.createDirectories;
import static java.nio.file.Files.walkFileTree;
import static com.google.common.io.ByteStreams.toByteArray;
import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.io.UncheckedIOException;
import java.io.Writer;
import java.nio.file.FileVisitResult;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import org.objectweb.asm.ClassReader;
import org.sonatype.aether.artifact.Artifact;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
/**
* The role of this class is to load the plugin class during development
*/
final class DolphinPluginDiscovery {
private static final String JAVA_CLASS_FILE_SUFFIX = ".class";
private static final String PLUGIN_SERVICES_FILE = "META-INF/services/" + DolphinSchedulerPlugin.class.getName();
private DolphinPluginDiscovery() {
}
public static Set<String> discoverPluginsFromArtifact(Artifact artifact, ClassLoader classLoader)
throws IOException {
if (!artifact.getExtension().equals("dolphinscheduler-plugin")) {
throw new RuntimeException("Unexpected extension for main artifact: " + artifact);
}
File file = artifact.getFile();
if (!file.getPath().endsWith("/target/classes")) {
throw new RuntimeException("Unexpected file for main artifact: " + file);
}
if (!file.isDirectory()) {
throw new RuntimeException("Main artifact file is not a directory: " + file);
}
if (new File(file, PLUGIN_SERVICES_FILE).exists()) {
return ImmutableSet.of();
}
return listClasses(file.toPath()).stream()
.filter(name -> classInterfaces(name, classLoader).contains(DolphinSchedulerPlugin.class.getName()))
.collect(Collectors.toSet());
}
public static void writePluginServices(Iterable<String> plugins, File root)
throws IOException {
Path path = root.toPath().resolve(PLUGIN_SERVICES_FILE);
createDirectories(path.getParent());
try (Writer out = new OutputStreamWriter(new FileOutputStream(path.toFile()), UTF_8)) {
for (String plugin : plugins) {
out.write(plugin + "\n");
}
}
}
private static List<String> listClasses(Path base)
throws IOException {
ImmutableList.Builder<String> list = ImmutableList.builder();
walkFileTree(base, new SimpleFileVisitor<Path>() {
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) {
if (file.getFileName().toString().endsWith(JAVA_CLASS_FILE_SUFFIX)) {
String name = file.subpath(base.getNameCount(), file.getNameCount()).toString();
list.add(javaName(name.substring(0, name.length() - JAVA_CLASS_FILE_SUFFIX.length())));
}
return FileVisitResult.CONTINUE;
}
});
return list.build();
}
private static List<String> classInterfaces(String name, ClassLoader classLoader) {
ImmutableList.Builder<String> list = ImmutableList.builder();
ClassReader reader = readClass(name, classLoader);
for (String binaryName : reader.getInterfaces()) {
list.add(javaName(binaryName));
}
if (reader.getSuperName() != null) {
list.addAll(classInterfaces(javaName(reader.getSuperName()), classLoader));
}
return list.build();
}
private static ClassReader readClass(String name, ClassLoader classLoader) {
try (InputStream in = classLoader.getResourceAsStream(binaryName(name) + JAVA_CLASS_FILE_SUFFIX)) {
if (in == null) {
throw new RuntimeException("Failed to read class: " + name);
}
return new ClassReader(toByteArray(in));
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
private static String binaryName(String javaName) {
return javaName.replace('.', '/');
}
private static String javaName(String binaryName) {
return binaryName.replace('/', '.');
}
}
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import static java.lang.String.format;
import static java.util.Objects.requireNonNull;
import static com.google.common.base.Preconditions.checkState;
import org.apache.dolphinscheduler.spi.DolphinSchedulerPlugin;
import org.apache.dolphinscheduler.spi.classloader.ThreadContextClassLoader;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.ServiceLoader;
import java.util.Set;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.sonatype.aether.artifact.Artifact;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Ordering;
import io.airlift.resolver.ArtifactResolver;
/**
* Plugin Loader
* Load Plugin from pom when development and run server in IDE
* Load Plugin from the plugin directory when running on the server
*/
public class DolphinPluginLoader {
private static final Logger logger = LoggerFactory.getLogger(DolphinPluginLoader.class);
/**
* All third-party jar packages used in the classes which in spi package need to be add
*/
private static final ImmutableList<String> DOLPHIN_SPI_PACKAGES = ImmutableList.<String>builder()
.add("org.apache.dolphinscheduler.spi.")
.add("com.fasterxml.jackson.")
.build();
private final File installedPluginsDir;
private final List<String> configPlugins;
private ArtifactResolver resolver = null;
private final List<AbstractDolphinPluginManager> dolphinPluginManagerList;
public DolphinPluginLoader(DolphinPluginManagerConfig config, List<AbstractDolphinPluginManager> dolphinPluginManagerList) {
installedPluginsDir = config.getInstalledPluginsDir();
if (config.getPlugins() == null) {
this.configPlugins = ImmutableList.of();
} else {
this.configPlugins = ImmutableList.copyOf(config.getPlugins());
}
this.dolphinPluginManagerList = requireNonNull(dolphinPluginManagerList, "dolphinPluginManagerList is null");
if (configPlugins != null && configPlugins.size() > 0) {
this.resolver = new ArtifactResolver(config.getMavenLocalRepository(), config.getMavenRemoteRepository());
}
}
public void loadPlugins()
throws Exception {
for (File file : listPluginDirs(installedPluginsDir)) {
if (file.isDirectory()) {
loadPlugin(file.getAbsolutePath());
}
}
for (String plugin : configPlugins) {
loadPlugin(plugin);
}
}
private void loadPlugin(String plugin)
throws Exception {
logger.info("-- Loading Alert plugin {} --", plugin);
URLClassLoader pluginClassLoader = buildPluginClassLoader(plugin);
try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(pluginClassLoader)) {
loadPlugin(pluginClassLoader);
}
logger.info("-- Finished loading Alert plugin {} --", plugin);
}
private void loadPlugin(URLClassLoader pluginClassLoader) {
ServiceLoader<DolphinSchedulerPlugin> serviceLoader = ServiceLoader.load(DolphinSchedulerPlugin.class, pluginClassLoader);
List<DolphinSchedulerPlugin> plugins = ImmutableList.copyOf(serviceLoader);
checkState(!plugins.isEmpty(), "No service providers the plugin {}", DolphinSchedulerPlugin.class.getName());
for (DolphinSchedulerPlugin plugin : plugins) {
logger.info("Installing {}", plugin.getClass().getName());
for (AbstractDolphinPluginManager dolphinPluginManager : dolphinPluginManagerList) {
dolphinPluginManager.installPlugin(plugin);
}
}
}
private URLClassLoader buildPluginClassLoader(String plugin)
throws Exception {
File file = new File(plugin);
if (!file.isDirectory() && (file.getName().equals("pom.xml") || file.getName().endsWith(".pom"))) {
return buildPluginClassLoaderFromPom(file);
}
if (file.isDirectory()) {
return buildPluginClassLoaderFromDirectory(file);
} else {
throw new IllegalArgumentException(format("plugin must be a pom file or directory {} .", plugin));
}
}
private URLClassLoader buildPluginClassLoaderFromPom(File pomFile)
throws Exception {
List<Artifact> artifacts = resolver.resolvePom(pomFile);
URLClassLoader classLoader = createClassLoader(artifacts, pomFile.getPath());
Artifact artifact = artifacts.get(0);
Set<String> plugins = DolphinPluginDiscovery.discoverPluginsFromArtifact(artifact, classLoader);
if (!plugins.isEmpty()) {
DolphinPluginDiscovery.writePluginServices(plugins, artifact.getFile());
}
return classLoader;
}
private URLClassLoader buildPluginClassLoaderFromDirectory(File dir)
throws Exception {
logger.info("Classpath for {}:", dir.getName());
List<URL> urls = new ArrayList<>();
for (File file : listPluginDirs(dir)) {
logger.info(" {}", file);
urls.add(file.toURI().toURL());
}
return createClassLoader(urls);
}
private URLClassLoader createClassLoader(List<Artifact> artifacts, String name)
throws IOException {
logger.info("Classpath for {}:", name);
List<URL> urls = new ArrayList<>();
for (Artifact artifact : sortArtifacts(artifacts)) {
if (artifact.getFile() == null) {
throw new RuntimeException("Could not resolve artifact: " + artifact);
}
File file = artifact.getFile().getCanonicalFile();
logger.info(" {}", file);
urls.add(file.toURI().toURL());
}
return createClassLoader(urls);
}
private URLClassLoader createClassLoader(List<URL> urls) {
ClassLoader parent = getClass().getClassLoader();
return new DolphinPluginClassLoader(urls, parent, DOLPHIN_SPI_PACKAGES);
}
private static List<File> listPluginDirs(File installedPluginsDir) {
if (installedPluginsDir != null && installedPluginsDir.isDirectory()) {
File[] files = installedPluginsDir.listFiles();
if (files != null) {
Arrays.sort(files);
return ImmutableList.copyOf(files);
}
}
return ImmutableList.of();
}
private static List<Artifact> sortArtifacts(List<Artifact> artifacts) {
List<Artifact> list = new ArrayList<>(artifacts);
Collections.sort(list, Ordering.natural().nullsLast().onResultOf(Artifact::getFile));
return list;
}
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import static java.lang.String.format;
import static java.util.Objects.requireNonNull;
import java.io.File;
import java.util.List;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableList;
/**
* Dolphin Scheduler Plugin Manager Config
*/
public class DolphinPluginManagerConfig {
/**
* The dir of the Alert Plugin in.
* When AlertServer is running on the server, it will load the Alert Plugin from this directory.
*/
private File installedPluginsDir;
/**
* The plugin should be load.
* The installedPluginsDir is empty when we development and run server in IDEA. Then we can config which plugin should be load by param name alert.plugin.binding in the alert.properties file
*/
private List<String> plugins;
/**
* Development, When AlertServer is running on IDE, AlertPluginLoad can load Alert Plugin from local Repository.
*/
private String mavenLocalRepository = System.getProperty("user.home") + "/.m2/repository";
private List<String> mavenRemoteRepository = ImmutableList.of("http://repo1.maven.org/maven2/");
public File getInstalledPluginsDir() {
return installedPluginsDir;
}
/**
* @param pluginDir
*/
public DolphinPluginManagerConfig setInstalledPluginsDir(String pluginDir) {
requireNonNull(pluginDir, "pluginDir can not be null");
File pluginDirFile = new File(pluginDir);
if (!pluginDirFile.exists()) {
throw new IllegalArgumentException(format("plugin dir not exists ! {}", pluginDirFile.getPath()));
}
this.installedPluginsDir = pluginDirFile;
return this;
}
public List<String> getPlugins() {
return plugins;
}
public DolphinPluginManagerConfig setPlugins(List<String> plugins) {
this.plugins = plugins;
return this;
}
/**
* When development and run server in IDE, this method can set plugins in alert.properties .
* Then when you start AlertServer in IDE, the plugin can be load.
* eg:
* file: alert.properties
* alert.plugin=\
* ../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml, \
* ../dolphinscheduler-alert-plugin/dolphinscheduler-alert-wechat/pom.xml
*
* @param plugins
* @return
*/
public DolphinPluginManagerConfig setPlugins(String plugins) {
if (plugins == null) {
this.plugins = null;
} else {
this.plugins = ImmutableList.copyOf(Splitter.on(',').omitEmptyStrings().trimResults().split(plugins));
}
return this;
}
public String getMavenLocalRepository() {
return mavenLocalRepository;
}
public DolphinPluginManagerConfig setMavenLocalRepository(String mavenLocalRepository) {
this.mavenLocalRepository = mavenLocalRepository;
return this;
}
public List<String> getMavenRemoteRepository() {
return mavenRemoteRepository;
}
public DolphinPluginManagerConfig setMavenRemoteRepository(List<String> mavenRemoteRepository) {
this.mavenRemoteRepository = mavenRemoteRepository;
return this;
}
public DolphinPluginManagerConfig setMavenRemoteRepository(String mavenRemoteRepository) {
this.mavenRemoteRepository = ImmutableList.copyOf(Splitter.on(',').omitEmptyStrings().trimResults().split(mavenRemoteRepository));
return this;
}
}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.plugin;
import org.apache.dolphinscheduler.alert.manager.DingTalkManager;
import org.apache.dolphinscheduler.alert.manager.EmailManager;
import org.apache.dolphinscheduler.alert.manager.EnterpriseWeChatManager;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.utils.DingTalkUtils;
import org.apache.dolphinscheduler.alert.utils.EnterpriseWeChatUtils;
import org.apache.dolphinscheduler.common.utils.CollectionUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.plugin.api.AlertPlugin;
import org.apache.dolphinscheduler.plugin.model.AlertData;
import org.apache.dolphinscheduler.plugin.model.AlertInfo;
import org.apache.dolphinscheduler.plugin.model.PluginName;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.*;
/**
* EmailAlertPlugin
*
* This plugin is a default plugin, and mix up email and enterprise wechat, because adapt with former alert behavior
*/
public class EmailAlertPlugin implements AlertPlugin {
private static final Logger logger = LoggerFactory.getLogger(EmailAlertPlugin.class);
private PluginName pluginName;
private static final EmailManager emailManager = new EmailManager();
private static final EnterpriseWeChatManager weChatManager = new EnterpriseWeChatManager();
private static final DingTalkManager dingTalkManager = new DingTalkManager();
public EmailAlertPlugin() {
this.pluginName = new PluginName();
this.pluginName.setEnglish(Constants.PLUGIN_DEFAULT_EMAIL_EN);
this.pluginName.setChinese(Constants.PLUGIN_DEFAULT_EMAIL_CH);
}
@Override
public String getId() {
return Constants.PLUGIN_DEFAULT_EMAIL_ID;
}
@Override
public PluginName getName() {
return pluginName;
}
@Override
@SuppressWarnings("unchecked")
public Map<String, Object> process(AlertInfo info) {
Map<String, Object> retMaps = new HashMap<>();
AlertData alert = info.getAlertData();
List<String> receiversList = (List<String>) info.getProp(Constants.PLUGIN_DEFAULT_EMAIL_RECEIVERS);
// receiving group list
// custom receiver
String receivers = alert.getReceivers();
if (StringUtils.isNotEmpty(receivers)) {
String[] splits = receivers.split(",");
receiversList.addAll(Arrays.asList(splits));
}
List<String> receiversCcList = new ArrayList<>();
// Custom Copier
String receiversCc = alert.getReceiversCc();
if (StringUtils.isNotEmpty(receiversCc)) {
String[] splits = receiversCc.split(",");
receiversCcList.addAll(Arrays.asList(splits));
}
if (CollectionUtils.isEmpty(receiversList) && CollectionUtils.isEmpty(receiversCcList)) {
logger.warn("alert send error : At least one receiver address required");
retMaps.put(Constants.STATUS, "false");
retMaps.put(Constants.MESSAGE, "execution failure,At least one receiver address required.");
return retMaps;
}
retMaps = emailManager.send(receiversList, receiversCcList, alert.getTitle(), alert.getContent(),
alert.getShowType());
//send flag
boolean flag = false;
if (retMaps == null) {
retMaps = new HashMap<>();
retMaps.put(Constants.MESSAGE, "alert send error.");
retMaps.put(Constants.STATUS, "false");
logger.info("alert send error : {}", retMaps.get(Constants.MESSAGE));
return retMaps;
}
flag = Boolean.parseBoolean(String.valueOf(retMaps.get(Constants.STATUS)));
if (flag) {
logger.info("alert send success");
retMaps.put(Constants.MESSAGE, "email send success.");
if (EnterpriseWeChatUtils.isEnable()) {
logger.info("Enterprise WeChat is enable!");
try {
String token = EnterpriseWeChatUtils.getToken();
weChatManager.send(info, token);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
if (DingTalkUtils.isEnableDingTalk) {
logger.info("Ding Talk is enable.");
dingTalkManager.send(info);
}
} else {
retMaps.put(Constants.MESSAGE, "alert send error.");
logger.info("alert send error : {}", retMaps.get(Constants.MESSAGE));
}
return retMaps;
}
}
......@@ -14,23 +14,24 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.runner;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.plugin.AlertPluginManager;
import org.apache.dolphinscheduler.common.enums.AlertStatus;
import org.apache.dolphinscheduler.common.plugin.PluginManager;
import org.apache.dolphinscheduler.dao.AlertDao;
import org.apache.dolphinscheduler.dao.PluginDao;
import org.apache.dolphinscheduler.dao.entity.Alert;
import org.apache.dolphinscheduler.dao.entity.User;
import org.apache.dolphinscheduler.plugin.api.AlertPlugin;
import org.apache.dolphinscheduler.plugin.model.AlertData;
import org.apache.dolphinscheduler.plugin.model.AlertInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.dolphinscheduler.dao.entity.AlertPluginInstance;
import org.apache.dolphinscheduler.spi.alert.AlertChannel;
import org.apache.dolphinscheduler.spi.alert.AlertData;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import org.apache.dolphinscheduler.spi.alert.AlertResult;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* alert sender
......@@ -41,57 +42,59 @@ public class AlertSender {
private List<Alert> alertList;
private AlertDao alertDao;
private PluginManager pluginManager;
private PluginDao pluginDao;
private AlertPluginManager alertPluginManager;
public AlertSender() {
public AlertSender(AlertPluginManager alertPluginManager) {
this.alertPluginManager = alertPluginManager;
}
public AlertSender(List<Alert> alertList, AlertDao alertDao, PluginManager pluginManager) {
public AlertSender(List<Alert> alertList, AlertDao alertDao, AlertPluginManager alertPluginManager, PluginDao pluginDao) {
super();
this.alertList = alertList;
this.alertDao = alertDao;
this.pluginManager = pluginManager;
this.pluginDao = pluginDao;
this.alertPluginManager = alertPluginManager;
}
public void run() {
List<User> users;
Map<String, Object> retMaps = null;
for (Alert alert : alertList) {
users = alertDao.listUserByAlertgroupId(alert.getAlertGroupId());
// receiving group list
List<String> receiversList = new ArrayList<>();
for (User user : users) {
receiversList.add(user.getEmail());
}
//get alert group from alert
int alertGroupId = alert.getAlertGroupId();
List<AlertPluginInstance> alertInstanceList = alertDao.listInstanceByAlertGroupId(alertGroupId);
AlertData alertData = new AlertData();
alertData.setId(alert.getId())
.setAlertGroupId(alert.getAlertGroupId())
.setContent(alert.getContent())
.setLog(alert.getLog())
.setReceivers(alert.getReceivers())
.setReceiversCc(alert.getReceiversCc())
.setShowType(alert.getShowType().getDescp())
.setTitle(alert.getTitle());
AlertInfo alertInfo = new AlertInfo();
alertInfo.setAlertData(alertData);
alertInfo.addProp("receivers", receiversList);
AlertPlugin emailPlugin = pluginManager.findOne(Constants.PLUGIN_DEFAULT_EMAIL_ID);
retMaps = emailPlugin.process(alertInfo);
if (retMaps == null) {
alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, "alert send error", alert.getId());
logger.error("alert send error : return value is null");
} else if (!Boolean.parseBoolean(String.valueOf(retMaps.get(Constants.STATUS)))) {
alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, String.valueOf(retMaps.get(Constants.MESSAGE)), alert.getId());
logger.error("alert send error : {}", retMaps.get(Constants.MESSAGE));
} else {
alertDao.updateAlert(AlertStatus.EXECUTION_SUCCESS, (String) retMaps.get(Constants.MESSAGE), alert.getId());
logger.info("alert send success");
for (AlertPluginInstance instance : alertInstanceList) {
String pluginName = pluginDao.getPluginDefineById(instance.getPluginDefineId()).getPluginName();
String pluginInstanceName = instance.getInstanceName();
AlertInfo alertInfo = new AlertInfo();
alertInfo.setAlertData(alertData);
alertInfo.setAlertParams(instance.getPluginInstanceParams());
AlertChannel alertChannel = alertPluginManager.getAlertChannelMap().get(pluginName);
if (alertChannel == null) {
alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, "Alert send error, not found plugin " + pluginName, alert.getId());
logger.error("Alert Plugin {} send error : not found plugin {}", pluginInstanceName, pluginName);
continue;
}
AlertResult alertResult = alertChannel.process(alertInfo);
if (alertResult == null) {
alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, "alert send error", alert.getId());
logger.info("Alert Plugin {} send error : return value is null", pluginInstanceName);
} else if (!Boolean.parseBoolean(String.valueOf(alertResult.getStatus()))) {
alertDao.updateAlert(AlertStatus.EXECUTION_FAILURE, String.valueOf(alertResult.getMessage()), alert.getId());
logger.info("Alert Plugin {} send error : {}", pluginInstanceName, alertResult.getMessage());
} else {
alertDao.updateAlert(AlertStatus.EXECUTION_SUCCESS, alertResult.getMessage(), alert.getId());
logger.info("Alert Plugin {} send success", pluginInstanceName);
}
}
}
......
......@@ -14,6 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
/**
......@@ -23,11 +24,15 @@ public class Constants {
private Constants() {
throw new IllegalStateException("Constants class");
}
/**
* alert properties path
*/
public static final String ALERT_PROPERTIES_PATH = "/alert.properties";
/** default alert plugin dir **/
public static final String ALERT_PLUGIN_PATH = "./lib/plugin/alert";
public static final String DATA_SOURCE_PROPERTIES_PATH = "/dao/data_source.properties";
public static final String SINGLE_SLASH = "/";
......@@ -41,41 +46,7 @@ public class Constants {
public static final String MESSAGE = "message";
public static final String MAIL_PROTOCOL = "mail.protocol";
public static final String MAIL_SERVER_HOST = "mail.server.host";
public static final String MAIL_SERVER_PORT = "mail.server.port";
public static final String MAIL_SENDER = "mail.sender";
public static final String MAIL_USER = "mail.user";
public static final String MAIL_PASSWD = "mail.passwd";
public static final String XLS_FILE_PATH = "xls.file.path";
public static final String MAIL_HOST = "mail.smtp.host";
public static final String MAIL_PORT = "mail.smtp.port";
public static final String MAIL_SMTP_AUTH = "mail.smtp.auth";
public static final String MAIL_TRANSPORT_PROTOCOL = "mail.transport.protocol";
public static final String MAIL_SMTP_STARTTLS_ENABLE = "mail.smtp.starttls.enable";
public static final String MAIL_SMTP_SSL_ENABLE = "mail.smtp.ssl.enable";
public static final String MAIL_SMTP_SSL_TRUST="mail.smtp.ssl.trust";
public static final String TEXT_HTML_CHARSET_UTF_8 = "text/html;charset=utf-8";
public static final String STRING_TRUE = "true";
public static final String EXCEL_SUFFIX_XLS = ".xls";
public static final int NUMBER_1000 = 1000;
public static final String SPRING_DATASOURCE_DRIVER_CLASS_NAME = "spring.datasource.driver-class-name";
......@@ -174,25 +145,11 @@ public class Constants {
public static final String DINGTALK_ENABLE = "dingtalk.isEnable";
public static final String HTML_HEADER_PREFIX = "<!DOCTYPE HTML PUBLIC '-//W3C//DTD HTML 4.01 Transitional//EN' 'http://www.w3.org/TR/html4/loose.dtd'><html><head><title>dolphinscheduler</title><meta name='Keywords' content=''><meta name='Description' content=''><style type=\"text/css\">table {margin-top:0px;padding-top:0px;border:1px solid;font-size: 14px;color: #333333;border-width: 1px;border-color: #666666;border-collapse: collapse;}table th {border-width: 1px;padding: 8px;border-style: solid;border-color: #666666;background-color: #dedede;text-align: left;}table td {border-width: 1px;padding: 8px;border-style: solid;border-color: #666666;background-color: #ffffff;text-align: left;}</style></head><body style=\"margin:0;padding:0\"><table border=\"1px\" cellpadding=\"5px\" cellspacing=\"-10px\"> ";
public static final String TABLE_BODY_HTML_TAIL = "</table></body></html>";
/**
* plugin config
*/
public static final String PLUGIN_DIR = "plugin.dir";
public static final String PLUGIN_DEFAULT_EMAIL_ID = "email";
public static final String PLUGIN_DEFAULT_EMAIL_CH = "邮件";
public static final String PLUGIN_DEFAULT_EMAIL_EN = "email";
public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERS = "receivers";
public static final String PLUGIN_DEFAULT_EMAIL_RECEIVERCCS = "receiverCcs";
public static final String RETMAP_MSG = "msg";
}
......@@ -14,10 +14,11 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.*;
import org.apache.commons.codec.binary.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
......@@ -32,13 +33,14 @@ import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* DingTalkUtils utils
* support send msg to ding talk by robot message push function.
......@@ -59,9 +61,10 @@ public class DingTalkUtils {
/**
* send message interface
* only support text message format now.
* @param msg message context to send
*
* @param msg message context to send
* @param charset charset type
* @return result of sending msg
* @return result of sending msg
* @throws IOException the IOException
*/
public static String sendDingTalkMsg(String msg, String charset) throws IOException {
......@@ -89,20 +92,19 @@ public class DingTalkUtils {
}
logger.info("Ding Talk send [{}], resp:{%s}", msg, resp);
return resp;
} finally {
} finally {
httpClient.close();
}
}
public static HttpPost constructHttpPost(String msg, String charset) {
HttpPost post = new HttpPost(dingTaskUrl);
HttpPost post = new HttpPost(dingTaskUrl);
StringEntity entity = new StringEntity(msg, charset);
post.setEntity(entity);
post.addHeader("Content-Type", "application/json; charset=utf-8");
return post;
}
public static CloseableHttpClient getProxyClient() {
HttpHost httpProxy = new HttpHost(proxy, port);
CredentialsProvider provider = new BasicCredentialsProvider();
......
......@@ -17,10 +17,12 @@
package org.apache.dolphinscheduler.alert.utils;
import org.apache.dolphinscheduler.common.enums.ShowType;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.apache.dolphinscheduler.plugin.model.AlertData;
import org.apache.dolphinscheduler.spi.alert.AlertConstants;
import org.apache.dolphinscheduler.spi.alert.AlertInfo;
import org.apache.dolphinscheduler.spi.alert.ShowType;
import org.apache.dolphinscheduler.spi.params.PluginParamsTransfer;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
......@@ -119,7 +121,7 @@ public class EnterpriseWeChatUtils {
*
* @param toParty the toParty
* @param agentId the agentId
* @param msg the msg
* @param msg the msg
* @return Enterprise WeChat send message
*/
public static String makeTeamSendMsg(String toParty, String agentId, String msg) {
......@@ -133,7 +135,7 @@ public class EnterpriseWeChatUtils {
*
* @param toParty the toParty
* @param agentId the agentId
* @param msg the msg
* @param msg the msg
* @return Enterprise WeChat send message
*/
public static String makeTeamSendMsg(Collection<String> toParty, String agentId, String msg) {
......@@ -146,9 +148,9 @@ public class EnterpriseWeChatUtils {
/**
* make team single user message
*
* @param toUser the toUser
* @param toUser the toUser
* @param agentId the agentId
* @param msg the msg
* @param msg the msg
* @return Enterprise WeChat send message
*/
public static String makeUserSendMsg(String toUser, String agentId, String msg) {
......@@ -160,9 +162,9 @@ public class EnterpriseWeChatUtils {
/**
* make team multi user message
*
* @param toUser the toUser
* @param toUser the toUser
* @param agentId the agentId
* @param msg the msg
* @param msg the msg
* @return Enterprise WeChat send message
*/
public static String makeUserSendMsg(Collection<String> toUser, String agentId, String msg) {
......@@ -176,8 +178,8 @@ public class EnterpriseWeChatUtils {
* send Enterprise WeChat
*
* @param charset the charset
* @param data the data
* @param token the token
* @param data the data
* @param token the token
* @return Enterprise WeChat resp, demo: {"errcode":0,"errmsg":"ok","invaliduser":""}
* @throws IOException the IOException
*/
......@@ -208,7 +210,7 @@ public class EnterpriseWeChatUtils {
/**
* convert table to markdown style
*
* @param title the title
* @param title the title
* @param content the content
* @return markdown table content
*/
......@@ -238,7 +240,7 @@ public class EnterpriseWeChatUtils {
/**
* convert text to markdown style
*
* @param title the title
* @param title the title
* @param content the content
* @return markdown text
*/
......@@ -272,12 +274,14 @@ public class EnterpriseWeChatUtils {
*
* @return the markdown alert table/text
*/
public static String markdownByAlert(AlertData alert) {
public static String markdownByAlert(AlertInfo alertInfo) {
String result = "";
if (alert.getShowType().equals(ShowType.TABLE.getDescp())) {
result = markdownTable(alert.getTitle(), alert.getContent());
} else if (alert.getShowType().equals(ShowType.TEXT.getDescp())) {
result = markdownText(alert.getTitle(), alert.getContent());
Map<String, String> paramsMap = PluginParamsTransfer.getPluginParamsMap(alertInfo.getAlertParams());
String showType = paramsMap.get(AlertConstants.SHOW_TYPE);
if (showType.equals(ShowType.TABLE.getDescp())) {
result = markdownTable(alertInfo.getAlertData().getTitle(), alertInfo.getAlertData().getContent());
} else if (showType.equals(ShowType.TEXT.getDescp())) {
result = markdownText(alertInfo.getAlertData().getTitle(), alertInfo.getAlertData().getContent());
}
return result;
......
......@@ -14,6 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
......@@ -22,7 +23,7 @@ public class FuncUtils {
public static String mkString(Iterable<String> list, String split) {
if (null == list || StringUtils.isEmpty(split)){
if (null == list || StringUtils.isEmpty(split)) {
return null;
}
......
......@@ -14,19 +14,21 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.utils;
import static org.apache.dolphinscheduler.alert.utils.Constants.ALERT_PROPERTIES_PATH;
import org.apache.dolphinscheduler.common.utils.IOUtils;
import org.apache.dolphinscheduler.common.utils.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import java.util.regex.PatternSyntaxException;
import static org.apache.dolphinscheduler.alert.utils.Constants.ALERT_PROPERTIES_PATH;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* property utils
......@@ -43,11 +45,11 @@ public class PropertyUtils {
private static final PropertyUtils propertyUtils = new PropertyUtils();
private PropertyUtils(){
private PropertyUtils() {
init();
}
private void init(){
private void init() {
String[] propertyFiles = new String[]{ALERT_PROPERTIES_PATH};
for (String fileName : propertyFiles) {
InputStream fis = null;
......@@ -69,6 +71,7 @@ public class PropertyUtils {
/**
* get property value
*
* @param key property name
* @return the value
*/
......@@ -82,7 +85,7 @@ public class PropertyUtils {
/**
* get property value
*
* @param key property name
* @param key property name
* @param defaultVal default value
* @return property value
*/
......@@ -95,7 +98,7 @@ public class PropertyUtils {
* get property value
*
* @param key property name
* @return get property int value , if key == null, then return -1
* @return get property int value , if key == null, then return -1
*/
public static int getInt(String key) {
......@@ -104,7 +107,8 @@ public class PropertyUtils {
/**
* get int value
* @param key the key
*
* @param key the key
* @param defaultValue the default value
* @return the value related the key or the default value if the key not existed
*/
......@@ -117,15 +121,16 @@ public class PropertyUtils {
try {
return Integer.parseInt(value);
} catch (NumberFormatException e) {
logger.info(e.getMessage(),e);
logger.info(e.getMessage(), e);
}
return defaultValue;
}
/**
* get property value
*
* @param key property name
* @return the boolean result value
* @return the boolean result value
*/
public static Boolean getBoolean(String key) {
......@@ -134,7 +139,7 @@ public class PropertyUtils {
}
String value = properties.getProperty(key.trim());
if(null != value){
if (null != value) {
return Boolean.parseBoolean(value);
}
......@@ -143,16 +148,18 @@ public class PropertyUtils {
/**
* get long value
*
* @param key the key
* @return if the value not existed, return -1, or will return the related value
*/
public static long getLong(String key) {
return getLong(key,-1);
return getLong(key, -1);
}
/**
* get long value
* @param key the key
*
* @param key the key
* @param defaultVal the default value
* @return the value related the key or the default value if the key not existed
*/
......@@ -166,7 +173,7 @@ public class PropertyUtils {
try {
return Long.parseLong(val);
} catch (NumberFormatException e) {
logger.info(e.getMessage(),e);
logger.info(e.getMessage(), e);
}
return defaultVal;
......@@ -174,17 +181,19 @@ public class PropertyUtils {
/**
* get double value
*
* @param key the key
* @return if the value not existed, return -1.0, or will return the related value
*/
public static double getDouble(String key) {
String val = getString(key);
return getDouble(key,-1.0);
return getDouble(key, -1.0);
}
/**
* get double value
* @param key the key
*
* @param key the key
* @param defaultVal the default value
* @return the value related the key or the default value if the key not existed
*/
......@@ -198,17 +207,17 @@ public class PropertyUtils {
try {
return Double.parseDouble(val);
} catch (NumberFormatException e) {
logger.info(e.getMessage(),e);
logger.info(e.getMessage(), e);
}
return defaultVal;
}
/**
* get array
* @param key property name
* @param splitStr separator
* get array
*
* @param key property name
* @param splitStr separator
* @return the result array
*/
public static String[] getArray(String key, String splitStr) {
......@@ -219,21 +228,22 @@ public class PropertyUtils {
try {
return value.split(splitStr);
} catch (PatternSyntaxException e) {
logger.info(e.getMessage(),e);
logger.info(e.getMessage(), e);
}
return null;
}
/**
* get enum
* @param key the key
* @param type the class type
*
* @param key the key
* @param type the class type
* @param defaultValue the default value
* @param <T> the generic class type
* @return get enum value
* @param <T> the generic class type
* @return get enum value
*/
public static <T extends Enum<T>> T getEnum(String key, Class<T> type,
T defaultValue) {
T defaultValue) {
String val = getString(key);
if (val == null) {
return defaultValue;
......@@ -242,7 +252,7 @@ public class PropertyUtils {
try {
return Enum.valueOf(type, val);
} catch (IllegalArgumentException e) {
logger.info(e.getMessage(),e);
logger.info(e.getMessage(), e);
}
return defaultValue;
......
......@@ -15,46 +15,19 @@
# limitations under the License.
#
#alert type is EMAIL/SMS
alert.type=EMAIL
#This configuration file configures the configuration parameters related to the AlertServer.
#These parameters are only related to the AlertServer, and it has nothing to do with the specific Alert Plugin.
#eg : max retry num.
#eg : Alert Server Listener port
# mail server configuration
mail.protocol=SMTP
mail.server.host=xxx.xxx.com
mail.server.port=25
mail.sender=xxx@xxx.com
mail.user=xxx@xxx.com
mail.passwd=111111
# TLS
mail.smtp.starttls.enable=true
# SSL
mail.smtp.ssl.enable=false
mail.smtp.ssl.trust=xxx.xxx.com
#alert.plugin.dir config the Alert Plugin dir . AlertServer while find and load the Alert Plugin Jar from this dir when deploy and start AlertServer on the server .
#eg :
#alert.plugin.dir=/root/dolphinscheduler/lib/plugin/alert
#xls file path,need create if not exist
#xls.file.path=/tmp/xls
#maven.local.repository=/Users/gaojun/Documents/jianguoyun/localRepository
# Enterprise WeChat configuration
enterprise.wechat.enable=false
#alert.plugin.binding config the Alert Plugin need be load when development and run in IDE
#alert.plugin.binding=\
# ./dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml
#enterprise.wechat.corp.id=xxxxxxx
#enterprise.wechat.secret=xxxxxxx
#enterprise.wechat.agent.id=xxxxxxx
#enterprise.wechat.users=xxxxxxx
#enterprise.wechat.token.url=https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid={corpId}&corpsecret={secret}
#enterprise.wechat.push.url=https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token={token}
#enterprise.wechat.team.send.msg={\"toparty\":\"{toParty}\",\"agentid\":\"{agentId}\",\"msgtype\":\"text\",\"text\":{\"content\":\"{msg}\"},\"safe\":\"0\"}
#enterprise.wechat.user.send.msg={\"touser\":\"{toUser}\",\"agentid\":\"{agentId}\",\"msgtype\":\"markdown\",\"markdown\":{\"content\":\"{msg}\"}}
plugin.dir=/Users/xx/your/path/to/plugin/dir
#ding talk configuration
dingtalk.isEnable=flase
dingtalk.webhook=https://oapi.dingtalk.com/robot/send?access_token=xxxxx
dingtalk.keyword=
dingtalk.proxy=
dingtalk.port=80
dingtalk.user=
dingtalk.password=
dingtalk.isEnableProxy=false
......@@ -14,52 +14,50 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.alert.template;
import org.apache.dolphinscheduler.alert.template.impl.DefaultHTMLTemplate;
package org.apache.dolphinscheduler.alert.plugin;
import org.apache.dolphinscheduler.alert.AlertServer;
import org.apache.dolphinscheduler.alert.utils.Constants;
import org.apache.dolphinscheduler.alert.utils.PropertyUtils;
import org.apache.dolphinscheduler.spi.utils.StringUtils;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.mockito.Mockito.*;
import static org.junit.Assert.*;
import com.google.common.collect.ImmutableList;
/**
* test class for AlertTemplateFactory
* AlertPluginManager Tester.
*/
@RunWith(PowerMockRunner.class)
@PrepareForTest(PropertyUtils.class)
public class AlertTemplateFactoryTest {
public class AlertPluginManagerTest {
private static final Logger logger = LoggerFactory.getLogger(AlertTemplateFactoryTest.class);
private static final Logger logger = LoggerFactory.getLogger(AlertPluginManagerTest.class);
/**
* GetMessageTemplate method test
*/
@Test
public void testGetMessageTemplate(){
PowerMockito.mockStatic(PropertyUtils.class);
AlertTemplate defaultTemplate = AlertTemplateFactory.getMessageTemplate();
assertTrue(defaultTemplate instanceof DefaultHTMLTemplate);
}
/**
* GetMessageTemplate method throw Exception test
*/
@Test
public void testGetMessageTemplateException(){
AlertTemplate defaultTemplate = AlertTemplateFactory.getMessageTemplate();
assertTrue(defaultTemplate instanceof DefaultHTMLTemplate);
public void testLoadPlugins() throws Exception {
logger.info("begin test AlertPluginManagerTest");
AlertPluginManager alertPluginManager = new AlertPluginManager();
DolphinPluginManagerConfig alertPluginManagerConfig = new DolphinPluginManagerConfig();
String path = DolphinPluginLoader.class.getClassLoader().getResource("").getPath();
alertPluginManagerConfig.setPlugins(path + "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml");
if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR))) {
alertPluginManagerConfig.setInstalledPluginsDir(org.apache.dolphinscheduler.alert.utils.PropertyUtils.getString(AlertServer.ALERT_PLUGIN_DIR, Constants.ALERT_PLUGIN_PATH).trim());
}
if (StringUtils.isNotBlank(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY))) {
alertPluginManagerConfig.setMavenLocalRepository(PropertyUtils.getString(AlertServer.MAVEN_LOCAL_REPOSITORY).trim());
}
DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager));
try {
alertPluginLoader.loadPlugins();
} catch (Exception e) {
throw new RuntimeException("load Alert Plugin Failed !", e);
}
Assert.assertNotNull(alertPluginManager.getAlertChannelFactoryMap().get("email alert"));
}
}
......@@ -14,48 +14,45 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.common.plugin;
package org.apache.dolphinscheduler.alert.plugin;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import java.net.URL;
import static org.junit.Assert.*;
public class PluginClassLoaderTest {
import com.google.common.collect.ImmutableList;
private PluginClassLoader pluginClassLoader;
private ClassLoader parent;
/**
* DolphinPluginLoader Tester.
*/
public class DolphinPluginLoaderTest {
@Before
public void setUp() {
parent = Thread.currentThread().getContextClassLoader();
pluginClassLoader = new PluginClassLoader(
new URL[]{}, parent,
null, null);
public void before() throws Exception {
}
@Test
public void loadClassNull() {
Class clazz = null;
try {
clazz = pluginClassLoader.loadClass("java.lang.Object");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
assertEquals(null, clazz.getClassLoader());
@After
public void after() throws Exception {
}
/**
* Method: loadPlugins()
*/
@Test
public void loadClassApp() {
Class clazz = null;
public void testLoadPlugins() throws Exception {
AlertPluginManager alertPluginManager = new AlertPluginManager();
DolphinPluginManagerConfig alertPluginManagerConfig = new DolphinPluginManagerConfig();
String path = DolphinPluginLoader.class.getClassLoader().getResource("").getPath();
alertPluginManagerConfig.setPlugins(path + "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml");
DolphinPluginLoader alertPluginLoader = new DolphinPluginLoader(alertPluginManagerConfig, ImmutableList.of(alertPluginManager));
try {
clazz = pluginClassLoader.loadClass("org.apache.dolphinscheduler.common.Constants");
} catch (ClassNotFoundException e) {
e.printStackTrace();
alertPluginLoader.loadPlugins();
} catch (Exception e) {
throw new RuntimeException("load Alert Plugin Failed !", e);
}
assertEquals(parent, clazz.getClassLoader());
}
}
\ No newline at end of file
Assert.assertNotNull(alertPluginManager.getAlertChannelFactoryMap().get("email alert"));
}
}
......@@ -17,14 +17,14 @@
package org.apache.dolphinscheduler.alert.utils;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import java.util.Arrays;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class FuncUtilsTest {
......
......@@ -17,13 +17,18 @@
package org.apache.dolphinscheduler.alert.utils;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertSame;
import static org.junit.Assert.assertTrue;
import org.apache.dolphinscheduler.common.enums.ZKNodeType;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.junit.Assert.*;
/**
* Test PropertyUtils
* and the resource path is src/test/resources/alert.properties.
......@@ -196,19 +201,19 @@ public class PropertyUtilsTest {
public void testGetEnum() {
//Expected MASTER
ZKNodeType zkNodeType = PropertyUtils.getEnum("test.server.enum1", ZKNodeType.class,ZKNodeType.WORKER);
ZKNodeType zkNodeType = PropertyUtils.getEnum("test.server.enum1", ZKNodeType.class, ZKNodeType.WORKER);
assertEquals(zkNodeType, ZKNodeType.MASTER);
//Expected DEAD_SERVER
zkNodeType = PropertyUtils.getEnum("test.server.enum2", ZKNodeType.class,ZKNodeType.WORKER);
zkNodeType = PropertyUtils.getEnum("test.server.enum2", ZKNodeType.class, ZKNodeType.WORKER);
assertEquals(zkNodeType, ZKNodeType.DEAD_SERVER);
//If key is null, then return defaultval
zkNodeType = PropertyUtils.getEnum(null, ZKNodeType.class,ZKNodeType.WORKER);
zkNodeType = PropertyUtils.getEnum(null, ZKNodeType.class, ZKNodeType.WORKER);
assertEquals(zkNodeType, ZKNodeType.WORKER);
//If the value doesn't define in enum ,it will log the error and return -1
zkNodeType = PropertyUtils.getEnum("test.server.enum3", ZKNodeType.class,ZKNodeType.WORKER);
zkNodeType = PropertyUtils.getEnum("test.server.enum3", ZKNodeType.class, ZKNodeType.WORKER);
assertEquals(zkNodeType, ZKNodeType.WORKER);
}
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册