From c44c6949c01e9b3b20fc96ed50210434736add7e Mon Sep 17 00:00:00 2001 From: Winnie Hellmann Date: Thu, 11 Oct 2018 12:37:34 +0200 Subject: [PATCH] Describe categories of frontend testing in testing guidelines --- .../new_fe_guide/development/testing.md | 268 ++++++++++++++++++ 1 file changed, 268 insertions(+) diff --git a/doc/development/new_fe_guide/development/testing.md b/doc/development/new_fe_guide/development/testing.md index a9223ac6b0f..d2aed2af1c3 100644 --- a/doc/development/new_fe_guide/development/testing.md +++ b/doc/development/new_fe_guide/development/testing.md @@ -1,5 +1,273 @@ # Overview of Frontend Testing +Tests relevant for frontend development can be found at two places: + +- `spec/javascripts/` which are run by Karma and contain + - [frontend unit tests](#frontend-unit-tests) + - [frontend component tests](#frontend-component-tests) + - [frontend integration tests](#frontend-integration-tests) +- `spec/features/` which are run by RSpec and contain + - [feature tests](#feature-tests) + +In addition there were feature tests in `features/` run by Spinach in the past. +These have been removed from our codebase in May 2018 ([#23036](https://gitlab.com/gitlab-org/gitlab-ce/issues/23036)). + +## Frontend unit tests + +Unit tests are on the lowest abstraction level and typically test functionality that is not directly perceivable by a user. + +### When to use unit tests + +
+ exported functions and classes + Anything that is exported can be reused at various places in a way you have no control over. + Therefore it is necessary to document the expected behavior of the public interface with tests. +
+ +
+ Vuex actions + Any Vuex action needs to work in a consistent way independent of the component it is triggered from. +
+ +
+ Vuex mutations + For complex Vuex mutations it helps to identify the source of a problem by separating the tests from other parts of the Vuex store. +
+ +### When *not* to use unit tests + +
+ non-exported functions or classes + Anything that is not exported from a module can be considered private or an implementation detail and doesn't need to be tested. +
+ +
+ constants + Testing the value of a constant would mean to copy it. + This results in extra effort without additional confidence that the value is correct. +
+ +
+ Vue components + Computed properties, methods, and lifecycle hooks can be considered an implementation detail of components and need not be tested. + They are implicitly covered by component tests. + The official guidelines suggest the same. +
+ +### What to mock in unit tests + +
+ state of the class under test + Modifying the state of the class under test directly rather than using methods of the class avoids side-effects in test setup. +
+ +
+ other exported classes + Every class needs to be tested in isolation to prevent test scenarios from growing exponentially. +
+ +
+ single DOM elements if passed as parameters + For tests that only operate on single DOM elements rather than a whole page, creating these elements is cheaper than loading a whole HTML fixture. +
+ +
+ all server requests + When running frontend unit tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +### What *not* to mock in unit tests + +
+ non-exported functions or classes + Everything that is not exported can be considered private to the module and will be implicitly tested via the exported classes / functions. +
+ +
+ methods of the class under test + By mocking methods of the class under test, the mocks will be tested and not the real methods. +
+ +
+ utility functions (pure functions, or those that only modify parameters) + If a function has no side effects because it has no state, it is safe to not mock it in tests. +
+ +
+ full HTML pages + Loading the HTML of a full page slows down tests, so it should be avoided in unit tests. +
+ +## Frontend component tests + +Component tests cover the state of a single component that is perceivable by a user depending on external signals such as user input, events fired from other components, or application state. + +### When to use component tests + +- Vue components + +### When *not* to use component tests + +
+ Vue applications + Vue applications may contain many components. + Testing them on a component level requires too much effort. + Therefore they are tested on frontend integration level. +
+ +
+ HAML templates + HAML templates contain only Markup and no frontend-side logic. + Therefore they are not complete components. +
+ +### What to mock in component tests + +
+ DOM + Operating on the real DOM is significantly slower than on the virtual DOM. +
+ +
+ properties and state of the component under test + Similarly to testing classes, modifying the properties directly (rather than relying on methods of the component) avoids side-effects. +
+ +
+ Vuex store + To avoid side effects and keep component tests simple, Vuex stores are replaced with mocks. +
+ +
+ all server requests + Similar to unit tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations + Similar to unit tests, background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +
+ child components + Every component is tested individually, so child components are mocked. + See also shallowMount() +
+ +### What *not* to mock in component tests + +
+ methods or computed properties of the component under test + By mocking part of the component under test, the mocks will be tested and not the real component. +
+ +
+ functions and classes independent from Vue + All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests. +
+ +## Frontend integration tests + +Integration tests cover the interaction between all components on a single page. +Their abstraction level is comparable to how a user would interact with the UI. + +### When to use integration tests + +
+ page bundles (index.js files in app/assets/javascripts/pages/) + Testing the page bundles ensures the corresponding frontend components integrate well. +
+ +
+ Vue applications outside of page bundles + Testing Vue applications as a whole ensures the corresponding frontend components integrate well. +
+ +### What to mock in integration tests + +
+ HAML views (use fixtures instead) + Rendering HAML views requires a Rails environment including a running database which we cannot rely on in frontend tests. +
+ +
+ all server requests + Similar to unit and component tests, when running component tests, the backend may not be reachable. + Therefore all outgoing requests need to be mocked. +
+ +
+ asynchronous background operations that are not perceivable on the page + Background operations that affect the page need to be tested on this level. + All other background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. +
+ +### What *not* to mock in integration tests + +
+ DOM + Testing on the real DOM ensures our components work in the environment they are meant for. + Part of this will be delegated to cross-browser testing. +
+ +
+ properties or state of components + On this level, all tests can only perform actions a user would do. + For example to change the state of a component, a click event would be fired. +
+ +
+ Vuex stores + When testing the frontend code of a page as a whole, the interaction between Vue components and Vuex stores is covered as well. +
+ +## Feature tests + +In contrast to [frontend integration tests](#frontend-integration-tests), feature tests make requests against the real backend instead of using fixtures. +This also implies that database queries are executed which makes this category significantly slower. + +### When to use feature tests + +- use cases that require a backend and cannot be tested using fixtures +- behavior that is not part of a page bundle but defined globally + +### Relevant notes + +A `:js` flag is added to the test to make sure the full environment is loaded. + +``` +scenario 'successfully', :js do + sign_in(create(:admin)) +end +``` + +The steps of each test are written using capybara methods ([documentation](http://www.rubydoc.info/gems/capybara/2.15.1)). + +Bear in mind XHR calls might require you to use `wait_for_requests` in between steps, like so: + +```rspec +find('.form-control').native.send_keys(:enter) + +wait_for_requests + +expect(page).not_to have_selector('.card') +``` + +## Test helpers + +--- + +> TODO: update the following sections + +--- + ## Types of tests in our codebase * **RSpec** -- GitLab