Release as a jar file, release updated user guide, peer-test released products, verify code authorship. Seek code quality comments from your tutor
v1.3 Summary of Milestone
| Milestone |
Minimum acceptable performance to consider as 'reached' |
| Contributed code to v1.3 |
code merged |
| Code is RepoSense-compatible |
as stated in mid-v1.3 |
| Generate project portfolio page |
extract parts of UG/DG into PPP. |
| v1.3 jar file released on GitHub |
as stated |
| v1.3 milestone properly wrapped up on GitHub |
as stated |
| Documentation updated to match v1.3 |
at least the User Guide and the README.adoc is updated |
v1.3 Project Management
- Ensure your code is RepoSense-compatible, as explained in mid-v1.3.
Using RepoSense
We recommend you ensure your code is RepoSense-compatible by v1.3
We use a tool called RepoSense to capture your contributions (i.e., each student's code) for grading.
Figure: RepoSense Report Features
1. View the current status of code authorship data:
- The report generated by the tool is available at Project Code Dashboard. The feature that is most relevant to you is the Code Panel (shown on the right side of the screenshot above). It shows the code attributed to a given author. You are welcome to play around with the other features.
- Click on your name to load the code attributed to you (based on Git blame/log data) onto the code panel on the right.
- If the code shown roughly matches the code you wrote, all is fine and there is nothing for you to do.
2. If the code does not match:
- Here are the possible reasons for the code shown not to match the code you wrote:
- the
git username in some of your commits does not match your GitHub username (perhaps you missed our instructions to set your Git username to match GitHub username earlier in the project, or GitHub did not honor your Git username for some reason)
- the actual authorship does not match the authorship determined by git blame/log e.g., another student touched your code after you wrote it, and Git log attributed the code to that student instead
3. You can run RepoSense locally:
- You can install reposense from the releases page, and generate the report locally to check.
- Use the two methods described in the RepoSense User Guide section Configuring a Repo to Provide Additional Data to RepoSense to provide additional data to the authorship analysis to make it more accurate.
- If you add a
config.json file to your repo (as specified by one of the two methods):
- If your commits have multiple author names, specify all of them e.g.,
"authorNames": ["theMyth", "theLegend", "theGary"]
- If updating the config file as described above results in an accurate report, pass the list of
authorNames to the lecturer so that the centralized dashboard can be updated to reflect the contributions accurately.
- If you add
@@author annotations, please follow the guidelines show below.
-
Mark your code with a //@@author {yourGithubUsername}. Note the double @.
The //@@author tag should indicates the beginning of the code you wrote. The code up to the next //@@author tag or the end of the file (whichever comes first) will be considered as was written by that author.
Here is a sample code file:
//@@author johndoe
method 1 ...
method 2 ...
//@@author sarahkhoo
method 3 ...
//@@author johndoe
method 4 ...
-
If you don't know who wrote the code segment below yours, you may put an empty //@@author (i.e. no GitHub username) to indicate the end of the code segment you wrote. The author of code below yours can add the GitHub username to the empty tag later.
Here is a sample code with an empty author tag:
method 0 ...
//@@author johndoe
method 1 ...
method 2 ...
//@@author
method 3 ...
method 4 ...
-
The author tag syntax varies based on file type e.g. for java, css, fxml. Use the corresponding comment syntax for non-Java files.
Here is an example code from an xml/fxml file.
<!-- @@author sereneWong -->
<textbox>
<label>...</label>
<input>...</input>
</textbox>
...
-
Do not put the //@@author inside java header comments.
👎
/**
* Returns true if ...
* @@author johndoe
*/
👍
//@@author johndoe
/**
* Returns true if ...
*/
What to and what not to annotate
-
Annotate both functional and test code There is no need to annotate documentation files.
-
Annotate only significant size code blocks that can be reviewed on its own e.g., a class, a sequence of methods, a method.
Claiming credit for code blocks smaller than a method is discouraged but allowed. If you do, do it sparingly and only claim meaningful blocks of code such as a block of statements, a loop, or an if-else statement.
- If an enhancement required you to do tiny changes in many places, there is no need to annotate all those tiny changes; you can describe those changes in the Project Portfolio page instead.
- If a code block was touched by more than one person, either let the person who wrote most of it (e.g. more than 80%) take credit for the entire block, or leave it as 'unclaimed' (i.e., no author tags).
- Related to the above point, if you claim a code block as your own, more than 80% of the code in that block should have been written by yourself. For example, no more than 20% of it can be code you reused from somewhere.
- 💡 GitHub has a blame feature and a history feature that can help you determine who wrote a piece of code.
-
Do not try to boost the quantity of your contribution using unethical means such as duplicating the same code in multiple places. In particular, do not copy-paste test cases to create redundant tests. Even repetitive code blocks within test methods should be extracted out as utility methods to reduce code duplication.
Individual members are responsible for making sure code attributed to them are correct.
If you notice a team member claiming credit for code that he/she did not write or use other questionable tactics, you can email us (after the final submission) to let us know.
-
If you wrote a significant amount of code that was not used in the final product,
- Create a folder called
{project root}/unused
- Move unused files (or copies of files containing unused code) to that folder
- use
//@@author {yourGithubUsername}-unused to mark unused code in those files (note the suffix unused)
e.g.
//@@author johndoe-unused
method 1 ...
method 2 ...
Please put a comment in the code to explain why it was not used.
-
If you reused code from elsewhere, mark such code as //@@author {yourGithubUsername}-reused (note the suffix reused)
e.g.
//@@author johndoe-reused
method 1 ...
method 2 ...
-
You can use empty @@author tags to mark code as not yours when RepoSense attribute the to you incorrectly.
-
Code generated by the IDE/framework, should not be annotated as your own.
-
Code you modified in minor ways e.g. adding a parameter. These should not be claimed as yours but you can mention these additional contributions in the Project Portfolio page if you want to claim credit for them.
v1.3 Product
v1.3 Documentation
v1.3 user guide should be updated to match the current version of the product. Reason: v1.3 will be subjected to a trial user testing session
- README page: Update to look like a real product if you haven't done so already. In particular, update the
Ui.png to match the current product (💡 tips).
-
User Guide: This document will be used by your peer testers. Update to match the current version. In particular,
- Clearly indicate which features are not implemented yet e.g. tag those features with a
Coming in v2.0.
- For those features already implemented, ensure their descriptions match the exact behavior of the product e.g. replace mockups with actual screenshots
-
Developer Guide: As before, update if necessary.
-
AboutUs page: Update to reflect current state of roles and responsibilities.
-
Product portfolio page: See below for details.
Submission: PDF submitted on LumiNUS.
Project Portfolio Page (PPP)
At the end of the project each student is required to submit a Project Portfolio Page.
-
Objective:
- For you to use (e.g. in your resume) as a well-documented data point of your SE experience
- For us to use as a data point to evaluate your,
- contributions to the project
- your documentation skills
-
Sections to include:
-
Overview: A short overview of your product (can use the product introduction you wrote earlier) to provide some context to the reader.
-
Summary of Contributions:
- Code contributed: Give a link to your code on Project Code Dashboard, which should be
https://nuscs2113-ay1920s1.github.io/dashboard/#=undefined&search=github_username_in_lower_case (replace github_username_in_lower_case with your actual username in lower case e.g., johndoe). This link is also available in the Project List Page -- linked to the icon under your photo.
- Features implemented: A summary of the features you implemented. If you implemented multiple features, you are recommended to indicate which one is the biggest feature.
- Other contributions:
- Contributions to project management e.g., setting up project tools, managing releases, managing issue tracker etc.
- Evidence of helping others e.g. responses you posted in our forum, bugs you reported in other team's products,
- Evidence of technical leadership e.g. sharing useful information in the forum
-
Relevant descriptions/terms/conventions: Include all relevant details necessary to understand the document, e.g., conventions, symbols or labels introduced by you, even if it was not introduced by you.
-
Contributions to the User Guide: Reproduce the parts in the User Guide that you wrote. This can include features you implemented as well as features you propose to implement.
The purpose of allowing you to include proposed features is to provide you more flexibility to show your documentation skills. e.g. you can bring in a proposed feature just to give you an opportunity to use a UML diagram type not used by the actual features.
-
Contributions to the Developer Guide: Reproduce the parts in the Developer Guide that you wrote. Ensure there is enough content to evaluate your technical documentation skills and UML modelling skills. You can include descriptions of your design/implementations, possible alternatives, pros and cons of alternatives, etc.
-
If you plan to use the PPP in your Resume, you can also include your SE work outside of the module (will not be graded)
-
Format:
-
File name: [TEAM_ID]-[Your Name]PPP.pdf e.g., [AY1920S1-CS2113T-F10-3][John Doe]PPP.pdf
-
Use one-half spacing between the lines for legibility
-
Follow this example. A PDF version of the same has been uploaded on LumiNUS.
-
💡 You are free to choose any (collaborative) software to write the documents. However, try to follow the format of the sample user guide, developer guide and PPP given.
-
Do note that extra effort is needed in duplicating and maintaining consistency across UG/DG and PPP. This is a cost of not using automated document generation.
-
It is assumed that all contents in the PPP were written primarily by you.
If any section is written by someone else e.g. someone else described the feature in the User Guide but you implemented the feature, clearly state that the section was written by someone else (e.g. Start of Extract [from: User Guide] written by Jane Doe). Reason: Your writing skills will be evaluated based on the PPP
-
Page limit:
| Content |
Limit |
| Overview + Summary of contributions |
0.5-1 (soft limit) |
| Contributions to the User Guide |
2-4 (soft limit) |
| Contributions to the Developer Guide |
3-5 (soft limit) |
| Total |
5-8 (strict) |
-
Reason for page limit: These submissions are peer-graded (in the PE) which needs to be done in a limited time span.
-
If you have more content than the limit given above, you can give a representative samples of UG and DG that showcase your documentation skills. Those samples should be understandable on their own. For the parts left-out, you can give an abbreviated version and refer the reader to the full UG/DG for more details.
-
It's similar to giving extra details as appendices; the reader will look at the UG/DG if the PPP is not enough to make a judgment. For example, when judging documentation quality, if the part in the PPP is not well-written, there is no point reading the rest in the main UG/DG. That's why you need to put the most representative part of your writings in the PPP and still give an abbreviated version of the rest in the PPP itself. Even when judging the quantity of work, the reader should be able to get a good sense of the quantity by combining what is quoted in the PPP and your abbreviated description of the missing part. There is no guarantee that the evaluator will read the full document.
v1.3 Demo
- Do a quick demo of the main features using the jar file. Objective: demonstrate that the jar file works.
v1.3 Testing (aka Practical Exam Dry Run)
Information updated on 30/10/2019
See info in the panel below:
Relevant: [Admin Project Deliverables → Practical Exam - Dry Run ]
What: The v1.3 is subjected to a round of peer acceptance/system testing, also called the Practical Exam Dry Run as this round of testing will be similar to the graded Practical Exam that will be done at v1.4.
When, where: uses a 45 minute slot at the end of week 11 lecture
Objectives:
- Evaluate your manual testing skills, product evaluation skills, effort estimation skills
- Peer-evaluate your product design , implementation effort , documentation quality
- Note that significant project components are not graded solely based on peer ratings. Rather, PE data are mostly used to cross-validate tutors' grades and identify cases that need further investigation. When peer inputs are used for grading, they are usually combined with tutor evaluations with appropriate weight for each. In some cases ratings from team members are given a higher weight compared to ratings from other peers, if that is appropriate.
- Do note that the PE is not a means of pitting you against each other. Developers and testers play for the same side; they need to push each other to improve the quality of their work -- not bring down each other.
Grading:
- Your performance in the practical exam will be considered for your final grade (under the QA category and under Implementation category, about 10 marks in total).
- You will be graded based on your effectiveness as a tester (e.g., the percentage of the bugs you found, the nature of the bugs you found) and how far off your evaluation/estimates are from the evaluator consensus. Explanation: we understand that you have limited expertise in this area; hence, we penalize only if your inputs don't seem to be based on a sincere effort to test/evaluate.
- The bugs found in your product by others will affect your v1.4 marks. You will be given a chance to reject false-positive bug reports.
Preparation:
- Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nusCS2113-AY1920S1 to accept the invitation.
- If you cannot find the invitation, post in our forum.
- Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.
- Download the latest CATcher and ensure you can run it on your computer.
- Create a public repo in your GitHub account with the following name:
- Enable its issue tracker and add the following labels to it (the label names should be precisely as given).
Bug Severity labels:
severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
Bug Type Labels:
type.FeatureFlaw: some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.type.FunctionalityBug: the bug is a flaw in how the product works.type.DocTypo: A minor spelling/grammar error in the documentation. Does not affect the user.type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction
-
Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.
- 💡 You can use Ctrl+V to paste a picture from the clipboard into a text box in GitHub issue tracker.
-
Download the product to be tested
after you have been notified of which team you have been allocated to test.
- Do this before you come to the testing session to minimize the network clogging due to large number of downloads happening at once.
-
Charge your computer before coming to the PE session. The testing venue (lecture theatre) does not have enough charging points.
Phase 1: Testing / bug reporting
When, where: Week 13 lecture
Testing
- Take note of your team to test. It will be given to you by the teaching team (distributed via LumiNUS gradebook).
- Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
- [60 minutes] Test the product and report bugs as described below:
Testing instructions for PE and PE Dry Run
Launching the JAR file
- Put the jar file in an empty folder.
- Open a command window. Run the java -version command to ensure you are using Java 11.
- Launch the jar file using the java -jar command (do not use double-clicking).
- If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, you can test the fallback team allocated to you. But in this case you must inform us immediately during/after the session so that we can send your bug reports to the correct team.
- PE-D: Download the JAR file from their GitHub page
- PE: Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
What to test:
- PE Dry Run (at v1.3):
- Test the product based on the User Guide (the UG is most likely accessible using the
help command).
- Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
- PE (at v1.4):
- Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
- As before, do both system testing and acceptance testing but give priority to system testing as system testing bugs can earn you more credit.
These are considered bugs:
- Behavior differs from the User Guide (or Developer Guide)
- A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
- Behavior is not specified and differs from normal expectations e.g. error message does not match the error
- The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
- Problems in the User Guide e.g., missing/incorrect info
About posting suggestions:
- PE Dry Run (at v1.3): You can also post suggestions on how to improve the product. 💡 Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
- PE (at v1.4): Do not post suggestions. But if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug.
How/where/when to report bugs:
-
Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bugs filed outside the testing duration will not be considered.
- Do not use team ID in bug reports. Reason: to prevent others copying your bug reports
- Launch CATcher, and login to the correct profile:
- PE Dry Run:
CS2113/T PE Dry run
- PE:
CS2113/T PE
- Post bugs using CATcher.
- Post bug reports in the following repo you created earlier:
Bug report format:
- Each bug should be a separate issue.
- Write good quality bug reports; poor quality (e.g., giving only screenshot with no description, details incomplete, etc.,) or incorrect bug reports will not earn credit.
- Use a descriptive title.
- Give a good description of the bug with steps to reproduce and screenshots.
- Assign exactly one
severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit).
Bug Severity labels:
severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
- Assign exactly one
type.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit).
Bug Type Labels:
type.FeatureFlaw: some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.type.FunctionalityBug: the bug is a flaw in how the product works.type.DocTypo: A minor spelling/grammar error in the documentation. Does not affect the user.type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction
Product evaluation
[Remainder of the session] Evaluate the following aspects. Note down your evaluation in a hard copy (as a backup). Submit via TEAMMATES. You are recommended to complete this during the PE session but you have until the end of the day to submit (or revise) your submissions.
-
A. Product Design []:
Evaluate the product design based on how the product V2.0 (not V1.4) is described in the User Guide.
-
B. Quality of user docs []:
Evaluate based on the parts of the user guide written by the person, as reproduced in the project portfolio. Evaluate from an end-user perspective.
-
C. Quality of developer docs []:
Evaluate based on the developer docs cited/reproduced in the respective project portfolio page. Evaluate from the perspective of a new developer trying to understand how the features are implemented.
-
D. Feature Quality []:
Evaluate the biggest feature done by the student for difficulty, completeness, and testability. Note: examples given below assume that AB3 did not have the commands edit, undo, and redo.
-
E. Amount of work []:
Evaluate the amount of work, on a scale of 0 to 30.
- Consider this PR (
history command) as 5 units of effort which means this PR (undo/redo command) is about 15 points of effort. Given that 30 points matches an effort twice as that needed for the undo/redo feature (which was given as an example of an A grade project), we expect most students to be have efforts lower than 20.
- Count all implementation/testing/documentation work as mentioned in that person's PPP. Also look at the actual code written by the person.
-
Do not give a high value just to be nice. If your estimate is wildly inaccurate, it means you are unable to estimate the effort required to implement a feature in a project that you are supposed to know well at this point. You may lose marks if your estimate is wildly inaccurate
Phase 2: Developer response phase
Duration: The review period will start around 1 day after the PE (exact time to be announced) and will last until the following Tuesday midnight. However, you are recommended to finish this task ASAP, to minimize cutting into your exam preparation work.
Bug reviewing is recommended to be done as a team as some of the decisions need team consensus.
Instructions for Reviewing Bug Reports
- First, don't freak out if there are lot of bug reports. Many can be duplicates and some can be false positives. In any case, we anticipate that all of these products will have some bugs and our penalty for bugs is not harsh. Furthermore, it depends on the severity of the bug. Some bug may not even be penalized.
- CATcher does not come with a UG, but the UI is fairly intuitive (there are tool tips too). Do post in the forum or ask in slack if you need any guidance with its usage.
- Also note that CATcher hasn't been battle-tested for this phase, in particular, w.r.t. multiple team members editing the same issue concurrently. It is ideal if the team members get together and work through the issues together. If you think others might be editing the same issues at the same time, use the
Sync button at the top to force-sync your view with the latest data from GitHub.
Launch CATcher, and login to the profile CS2113/T PE. It will show all the bugs assigned to your team, divided into three sections:
Issues Pending Responses - Issues that your team has not processed yet.Issues Responded - Your job is to get all issues to the second category.Faulty Issues - e.g., Bugs marked as duplicates of each other, or causing circular duplicate relationships. Fix the problem given so that no issues remain in this category.
Respond to the bug reports shown.
Issues created for PE-D and PE need to be in a precise format for our grading scripts to work. Incorrectly-formatted responses will have to discarded. Therefore, you are strongly recommended to use CATcher for PE-D and PE activities. If you want to give your response via GitHub instead, please get our permission first.
- Go to the dev-response issue tracker
- Use
tutorial.* and team.* labels to filter bug reports your team received.
- Do not edit the subject or the description. Your response (if any) should be added as a comment.
- Add a comment using the following exact template.
# Team's Response
{replace this with your response}
## Duplicate status (if any):
Here is an example:
# Team's Response
Yes this is a bug. But it is a duplicate.
* Changed the bug type because this is just a bug in the UG.
* Lowered the severity because users can still use the feature.
## Duplicate status (if any):
Duplicate of #67
- Do not close the bug report after you are done processing it.
- Use the exact
Duplicate of #123 format to indicate duplicates.
- There should be exactly one comment per issue. If there are multiple comments, the last one will be taken for processing.
-
If a bug seems to be for a different product (i.e. wrongly assigned to your team), let us know ASAP (email prof).
-
If the bug is reported multiple times,
- Mark all copies EXCEPT one as duplicates of the one left out (let's call that one the original) using the
duplicate tag.
- If the duplicates have different severity levels, you should keep the one with the highest severity as the original. But you can downgrade the severity of the original or the duplicates.
- For each group of duplicates, all duplicates should point to one original i.e., no multiple levels of duplicates, and no cyclical duplication relationships.
- If the duplication status is eventually accepted, all duplicates will be assumed to have inherited the
type.* and severity.* from the original.
-
Apply exactly one of these labels (if missing, we assign: response.Accepted)
Response Labels:
response.Accepted: You accept it as a bug.response.NotInScope: It is a valid issue but not something the team should be penalized for e.g., it was not related to features delivered in v1.4.response.Rejected: What tester treated as a bug is in fact the expected behavior. The may lose marks for rejecting a bug without an explanation or using an unjustifiable explanation.response.CannotReproduce: You are unable to reproduce the behavior reported in the bug after multiple tries.response.IssueUnclear: The issue description is not clear. Don't post comments asked the tester to give more info. The tester will not be able to see those comments because the bug reports are anonymized.
- Apply exactly one of these labels (if missing, we assign:
type.FunctionalityBug)
Bug Type Labels:
type.FeatureFlaw: some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.type.FunctionalityBug: the bug is a flaw in how the product works.type.DocTypo: A minor spelling/grammar error in the documentation. Does not affect the user.type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction
- If you disagree with the original severity assigned to the bug, you may change it to the correct level, in which case add a comment justifying the change. All such changes will be double-checked by the teaching team.
Bug Severity labels:
severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
-
Decide who should fix the bug. Use the Assignees field to assign the issue to that person(s). There is no need to actually fix the bug though. It's simply an indication/acceptance of responsibility. If there is no assignee, we will distribute the penalty for that bug (if any) among all team members.
- If it is not easy to decide the assignee(s), we recommend (but not enforce) that the feature owner should be assigned bugs related to the feature, Reason: The feature owner should have defended the feature against bugs using automated tests and defensive coding techniques.
-
As far as possible, choose the correct type.*, severity.*, and assignees even for bugs you are not accepting or for bugs that are marked as duplicates. Reason: your non-acceptance or duplication status may be rejected in a later phase, in which case we need to grade it as an accepted/non-duplicate bug.
- Justify your response. For all of the following cases, you must add a comment justifying your stance. Testers will get to respond to all those cases and will be double-checked by the teaching team in later phases. Indiscriminate/unreasonable dev/tester responses, if deemed as a case of trying to game the system, will be penalized.
- downgrading severity
- non-acceptance of a bug
- changing the bug type
- non-obvious duplicate
Grading: Taking part in the PE dry run is strongly encouraged as it can affect your grade in the following ways.
- If the product you are allocated to test in the Practical Exam (at v1.4) had a very low bug count, we will consider your performance in PE dry run as well when grading the PE.
- PE dry run will help you practice for the actual PE.
- Taking part in the PE dry run will earn you participation points.
- There is no penalty for bugs reported in your product. Every bug you find is a win-win for you and the team whose product you are testing.
Objectives:
- To train you to do manual testing, bug reporting, bug triaging, bug fixing, communicating with users/testers/developers, evaluating products etc.
- To help you improve your product before the final submission.
Preparation
- Ensure that you have accepted the invitation to join the GitHub org used by the module. Go to https://github.com/nusCS2113-AY1920S1 to accept the invitation.
- If you cannot find the invitation, post in our forum.
- Ensure you have access to a computer that is able to run module projects e.g. has the right Java version.
- Download the latest CATcher and ensure you can run it on your computer.
- Create a public repo in your GitHub account with the following name:
- Enable its issue tracker and add the following labels to it (the label names should be precisely as given).
Bug Severity labels:
severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
Bug Type Labels:
type.FeatureFlaw: some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.type.FunctionalityBug: the bug is a flaw in how the product works.type.DocTypo: A minor spelling/grammar error in the documentation. Does not affect the user.type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction
-
Have a good screen grab tool with annotation features so that you can quickly take a screenshot of a bug, annotate it, and post in the issue tracker.
- 💡 You can use Ctrl+V to paste a picture from the clipboard into a text box in GitHub issue tracker.
-
Download the product to be tested
after you have been notified of which team you have been allocated to test.
- Do this before you come to the testing session to minimize the network clogging due to large number of downloads happening at once.
-
Charge your computer before coming to the PE session. The testing venue (lecture theatre) does not have enough charging points.
During the session:
- Take note of your team to test. Distributed via LumiNUS gradebook.
- Download the latest jar file from the team's GitHub page (if you haven't done yet). Copy it to an empty folder.
- Confirm you are testing the allocated product by comparing the product UI with the UI screenshot seen in the project dashboard.
Testing instructions for PE and PE Dry Run
Launching the JAR file
- Put the jar file in an empty folder.
- Open a command window. Run the java -version command to ensure you are using Java 11.
- Launch the jar file using the java -jar command (do not use double-clicking).
- If the product doesn't work at all: If the product fails catastrophically e.g., cannot even launch, you can test the fallback team allocated to you. But in this case you must inform us immediately during/after the session so that we can send your bug reports to the correct team.
- PE-D: Download the JAR file from their GitHub page
- PE: Download from LumiNUS all files submitted by the team (i.e. jar file, User Guide, Developer Guide, and Project Portfolio Pages) into an empty folder.
What to test:
- PE Dry Run (at v1.3):
- Test the product based on the User Guide (the UG is most likely accessible using the
help command).
- Do system testing first i.e., does the product work as specified by the documentation?. If there is time left, you can do acceptance testing as well i.e., does the product solve the problem it claims to solve?.
- PE (at v1.4):
- Test based on the Developer Guide (Appendix named Instructions for Manual Testing) and the User Guide. The testing instructions in the Developer Guide can provide you some guidance but if you follow those instructions strictly, you are unlikely to find many bugs. You can deviate from the instructions to probe areas that are more likely to have bugs.
- As before, do both system testing and acceptance testing but give priority to system testing as system testing bugs can earn you more credit.
These are considered bugs:
- Behavior differs from the User Guide (or Developer Guide)
- A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
- Behavior is not specified and differs from normal expectations e.g. error message does not match the error
- The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
- Problems in the User Guide e.g., missing/incorrect info
About posting suggestions:
- PE Dry Run (at v1.3): You can also post suggestions on how to improve the product. 💡 Be diplomatic when reporting bugs or suggesting improvements. For example, instead of criticising the current behavior, simply suggest alternatives to consider.
- PE (at v1.4): Do not post suggestions. But if a feature is missing a critical functionality that makes the feature less useful to the intended user, it can be reported as a bug.
How/where/when to report bugs:
-
Post bugs as you find them (i.e., do not wait to post all bugs at the end) because bugs filed outside the testing duration will not be considered.
- Do not use team ID in bug reports. Reason: to prevent others copying your bug reports
- Launch CATcher, and login to the correct profile:
- PE Dry Run:
CS2113/T PE Dry run
- PE:
CS2113/T PE
- Post bugs using CATcher.
- Post bug reports in the following repo you created earlier:
Bug report format:
- Each bug should be a separate issue.
- Write good quality bug reports; poor quality (e.g., giving only screenshot with no description, details incomplete, etc.,) or incorrect bug reports will not earn credit.
- Use a descriptive title.
- Give a good description of the bug with steps to reproduce and screenshots.
- Assign exactly one
severity.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit).
Bug Severity labels:
severity.Low : A flaw that is unlikely to affect normal operations of the product. Appears only in very rare situations and causes a minor inconvenience only.severity.Medium : A flaw that causes occasional inconvenience to some users but they can continue to use the product.severity.High : A flaw that affects most users and causes major problems for users. i.e., makes the product almost unusable for most users.
- Assign exactly one
type.* label to the bug report. Bug report without a severity label are considered severity.Low (lower severity bugs earn lower credit).
Bug Type Labels:
type.FeatureFlaw: some functionality missing from a feature delivered in v1.4 in a way that the feature becomes less useful to the intended target user for normal usage. i.e., the feature is not 'complete'. In other words, an acceptance testing bug that falls within the scope of v1.4 features. These issues are counted against the 'depth and completeness' of the feature.type.FunctionalityBug: the bug is a flaw in how the product works.type.DocTypo: A minor spelling/grammar error in the documentation. Does not affect the user.type.DocumentationBug: A flaw in the documentation that can potentially affect the user e.g., a missing step, a wrong instruction
At the end of the project each student is required to submit a Project Portfolio Page.
-
Objective:
- For you to use (e.g. in your resume) as a well-documented data point of your SE experience
- For us to use as a data point to evaluate your,
- contributions to the project
- your documentation skills
-
Sections to include:
-
Overview: A short overview of your product (can use the product introduction you wrote earlier) to provide some context to the reader.
-
Summary of Contributions:
- Code contributed: Give a link to your code on Project Code Dashboard, which should be
https://nuscs2113-ay1920s1.github.io/dashboard/#=undefined&search=github_username_in_lower_case (replace github_username_in_lower_case with your actual username in lower case e.g., johndoe). This link is also available in the Project List Page -- linked to the icon under your photo.
- Features implemented: A summary of the features you implemented. If you implemented multiple features, you are recommended to indicate which one is the biggest feature.
- Other contributions:
- Contributions to project management e.g., setting up project tools, managing releases, managing issue tracker etc.
- Evidence of helping others e.g. responses you posted in our forum, bugs you reported in other team's products,
- Evidence of technical leadership e.g. sharing useful information in the forum
-
Relevant descriptions/terms/conventions: Include all relevant details necessary to understand the document, e.g., conventions, symbols or labels introduced by you, even if it was not introduced by you.
-
Contributions to the User Guide: Reproduce the parts in the User Guide that you wrote. This can include features you implemented as well as features you propose to implement.
The purpose of allowing you to include proposed features is to provide you more flexibility to show your documentation skills. e.g. you can bring in a proposed feature just to give you an opportunity to use a UML diagram type not used by the actual features.
-
Contributions to the Developer Guide: Reproduce the parts in the Developer Guide that you wrote. Ensure there is enough content to evaluate your technical documentation skills and UML modelling skills. You can include descriptions of your design/implementations, possible alternatives, pros and cons of alternatives, etc.
-
If you plan to use the PPP in your Resume, you can also include your SE work outside of the module (will not be graded)
-
Format:
-
File name: [TEAM_ID]-[Your Name]PPP.pdf e.g., [AY1920S1-CS2113T-F10-3][John Doe]PPP.pdf
-
Use one-half spacing between the lines for legibility
-
Follow this example. A PDF version of the same has been uploaded on LumiNUS.
-
💡 You are free to choose any (collaborative) software to write the documents. However, try to follow the format of the sample user guide, developer guide and PPP given.
-
Do note that extra effort is needed in duplicating and maintaining consistency across UG/DG and PPP. This is a cost of not using automated document generation.
-
It is assumed that all contents in the PPP were written primarily by you.
If any section is written by someone else e.g. someone else described the feature in the User Guide but you implemented the feature, clearly state that the section was written by someone else (e.g. Start of Extract [from: User Guide] written by Jane Doe). Reason: Your writing skills will be evaluated based on the PPP
-
Page limit:
| Content |
Limit |
| Overview + Summary of contributions |
0.5-1 (soft limit) |
| Contributions to the User Guide |
2-4 (soft limit) |
| Contributions to the Developer Guide |
3-5 (soft limit) |
| Total |
5-8 (strict) |
-
Reason for page limit: These submissions are peer-graded (in the PE) which needs to be done in a limited time span.
-
If you have more content than the limit given above, you can give a representative samples of UG and DG that showcase your documentation skills. Those samples should be understandable on their own. For the parts left-out, you can give an abbreviated version and refer the reader to the full UG/DG for more details.
-
It's similar to giving extra details as appendices; the reader will look at the UG/DG if the PPP is not enough to make a judgment. For example, when judging documentation quality, if the part in the PPP is not well-written, there is no point reading the rest in the main UG/DG. That's why you need to put the most representative part of your writings in the PPP and still give an abbreviated version of the rest in the PPP itself. Even when judging the quantity of work, the reader should be able to get a good sense of the quantity by combining what is quoted in the PPP and your abbreviated description of the missing part. There is no guarantee that the evaluator will read the full document.
After the session:
- We'll transfer the relevant bug reports to your repo over the weekend. Once you have received the bug reports for your product, it is up to you to decide whether you will act on reported issues before the final submission v1.4. For some issues, the correct decision could be to reject or postpone to a version beyond v1.4.
- You can navigate to the original bug report (via the back-link provided in the bug report given to you) and post in that issue thread to communicate with the tester who reported the bug e.g. to ask for more info, etc. However, the tester is not obliged to respond.
- 💡 Do not argue with the issue reporter to try to convince that person that your way is correct/better. If at all, you can gently explain the rationale for the current behavior but do not waste time getting involved in long arguments. If you think the suggestion/bug is unreasonable, just thank the reporter for their view and close the issue.
