Graph Attention Networks(GAT) was proposed in 2017 by Petar Veličković et al. By leveraging masked self-attentional layers to address shortcomings of prior graph based method, GAT achieved or matched state of the art performance on both transductive datasets like Cora and inductive dataset like PPI. This is an example of training GAT with Cora dataset in MindSpore.
Graph Attention Networks(GAT) was proposed in 2017 by Petar Veličković et al. By leveraging masked self-attentional layers to address shortcomings of prior graph based method, GAT achieved or matched state of the art performance on both transductive datasets like Cora and inductive dataset like PPI. This is an example of training GAT with Cora dataset in MindSpore.
[Paper](https://arxiv.org/abs/1710.10903): Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2017). Graph attention networks. arXiv preprint arXiv:1710.10903.
[Paper](https://arxiv.org/abs/1710.10903): Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2017). Graph attention networks. arXiv preprint arXiv:1710.10903.
# Model architecture
# [Model architecture](#contents)
An illustration of multi- head attention (with K = 3 heads) by node 1 on its neighborhood can be found below:
Download the dataset Cora or Citeseer provided by /kimiyoung/planetoid from github.
- Data Preparation
> Place the dataset to any path you want, the folder should include files as follows(we use Cora dataset as an example):
> Place the dataset to any path you want, the folder should include files as follows(we use Cora dataset as an example):
```
.
```
└─data
.
├─ind.cora.allx
└─data
├─ind.cora.ally
├─ind.cora.allx
├─ind.cora.graph
├─ind.cora.ally
├─ind.cora.test.index
├─ind.cora.graph
├─ind.cora.tx
├─ind.cora.test.index
├─ind.cora.ty
├─ind.cora.tx
├─ind.cora.x
├─ind.cora.ty
└─ind.cora.y
├─ind.cora.x
```
└─ind.cora.y
```
> Generate dataset in mindrecord format for cora or citeseer.
>> Usage
> Generate dataset in mindrecord format for cora or citeseer.
```buildoutcfg
>> Usage
cd ./scripts
```buildoutcfg
# SRC_PATH is the dataset file path you downloaded, DATASET_NAME is cora or citeseer
cd ./scripts
sh run_process_data_ascend.sh [SRC_PATH] [DATASET_NAME]
# SRC_PATH is the dataset file path you downloaded, DATASET_NAME is cora or citeseer
```
sh run_process_data.sh [SRC_PATH] [DATASET_NAME]
```
>> Launch
```
>> Launch
#Generate dataset in mindrecord format for cora
```
./run_process_data_ascend.sh ./data cora
#Generate dataset in mindrecord format for cora
#Generate dataset in mindrecord format for citeseer
./run_process_data.sh ./data cora
./run_process_data_ascend.sh ./data citeseer
#Generate dataset in mindrecord format for citeseer
```
./run_process_data.sh ./data citeseer
```
# [Features](#contents)
# Features
## Mixed Precision
## Mixed Precision
To ultilize the strong computation power of Ascend chip, and accelerate the training process, the mixed training method is used. MindSpore is able to cope with FP32 inputs and FP16 operators. In GAT example, the model is set to FP16 mode except for the loss calculation part.
To ultilize the strong computation power of Ascend chip, and accelerate the training process, the mixed training method is used. MindSpore is able to cope with FP32 inputs and FP16 operators. In GAT example, the model is set to FP16 mode except for the loss calculation part.
| Training Cost(200 epochs) | 27.62298311s | 36.711862s |
| End to End Training Cost(200 epochs) | 39.074s | 50.894s |
# Description of random situation
GAT model contains lots of dropout operations, if you want to disable dropout, set the attn_dropout and feature_dropout to 0 in src/config.py. Note that this operation will cause the accuracy drop to approximately 80%.
GAT model contains lots of dropout operations, if you want to disable dropout, set the attn_dropout and feature_dropout to 0 in src/config.py. Note that this operation will cause the accuracy drop to approximately 80%.
# Others
# [ModelZoo Homepage](#contents)
GAT model is verified on Ascend environment, not on CPU or GPU.
\ No newline at end of file
Please check the official [homepage](http://gitee.com/mindspore/mindspore/tree/master/model_zoo).
Cora and Citeseer datasets are supported by our example, the original versions of these datasets are as follows:
@inproceedings{nr,
title={The Network Data Repository with Interactive Graph Analytics and Visualization},
author={Ryan A. Rossi and Nesreen K. Ahmed},
booktitle={AAAI},
url={http://networkrepository.com},
year={2015}
}
In this example, we use dataset splits provided by https://github.com/kimiyoung/planetoid (Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov, [Revisiting Semi-Supervised Learning with Graph Embeddings](https://arxiv.org/abs/1603.08861), ICML 2016).
