Order from us for quality, customized work in due time of your choice.
Subject: The Effects of Including Co-Curricular Activities on Veteran Student Performance: Final Evaluation Design Plan.
The goal of this memo is to introduce an evaluation design plan to be applied to the Armed Services Arts Partnership (ASAP) program and the analysis of the effects of its co-curricular activities on student veterans. First, the discussion of the main veteran concepts, the program, and program theory will be developed within the literature review. The next step embraces the evaluation of goals and questions within this project.
Then, the description of a data collection plan and a data analysis plan will be given in order to create a definite background for working and communicating with stakeholders, including veterans and the founders of the ASAP program, through the prism of a properly developed data collection instrument. Program evaluation is a good chance to combine empirical evidence and real-life information about the current program and policies and identify their impact on ordinary people. There are many non-profit organizations that want to implement their approaches and increase the quality of human life. The idea to support veterans cannot be neglected, and the evaluation of the ASAP program has to be developed in accordance with a strong and well-prepared plan.
Role of Veterans in US Society
Todays US society is based on a number of important principles and rules, support for human rights and freedoms, democracy, respect for each other, and equality. In addition to this impressive list of virtues, the role of veterans cannot be neglected. US veterans introduce a population with certain values and codes of conduct that are determined by obedience, command, and order (Olenick, Flowers, & Diaz, 2015).
People of different ages are ready to demonstrate their attitudes towards veterans and their place in this word. For example, an 11th-grader, Caya Wollman (2014), published an article where she defined veterans as the backbone of the nation without which there would be no USA, life, and liberty. Her discussion promotes an understanding of veterans as a pledge of democracy and justice that have to be spread around the whole world. Therefore, the idea to provide veterans with an opportunity to study and improve their academic performance seems to be a normal contribution to the social and ethical development of the country.
Student Veterans and Their Concerns
The concept of a student veteran continues playing an important role in US society, and many steps are taken to promote the idea of higher education for people with such an experience. In 2009, the Post 9/11 GI Bill was introduced and promoted an education opportunity for more than 600,000 veterans (Griffin & Gilbert, 2015). At the same time, this decision helped many veterans receive full tuition, monthly housing payment, and the coverage of money expenses on books and other educational supplies (Callahan & Jarrat, 2014).
Within a short period of time, the quality of veteran life was considerably improved, and new co-curricular activities were introduced. However, certain concerns and challenges cannot be ignored in this field. For example, it is expected to learn the effect of higher education on the future of veterans and understand the urgency of academic activities for both veterans and communities where they have to live.
Co-Curricular Activities
Facilities that offer higher education and prepare veterans for their academic success continue developing in modern society. There are many subjects and projects with the frames of which veterans demonstrate their skills and define their potential hobbies. For example, the investigation of Steele (2015) shows that the majority of veterans are usually interested in such subjects as foreign languages, literature, mathematics, and laboratory experiments.
At the same time, many people want to develop their social skills and communication. The establishment of trustful relationships among people is one of the first priorities for the developers of co-curriculum activities for veterans. To promote the achievement of this goal, faculty presence is required during which people can share their emotions and experiences and observe the reactions of other veterans (Sutton, 2016). The exchange of information in real-life settings without guns and bullets disturbing their ordinary lives is an important part of human life, and veterans must learn how to develop this activity in a correct and less harmful way.
Programs for Veterans and Well-Being Improvement
Many academic programs are offered to student veterans within the existing system of higher education in the United States. Some programs are available to all the participants, and some courses have a serious criteria system according to which veterans should take tests and prove their appropriateness for a particular facility. In addition to the already mentioned GI Bill, the Department of Veteran Affairs supports the development of various non-profit organizations and programs.
Veteran Success Jam, Veterans for Education, and Student Veterans of America are the programs that inspire and help veterans get back to their education and have a chance to find a good job (Powers, 2018). Veterans well-being can be improved in terms of oral and writing abilities, critical thinking, communication skills, and decision-making. Each program has its focus, goals, and methods of education. This project helps evaluate the worth of one particular program for modern veterans in the United States.
ASAP Program
The ASAP program is available to veterans who live in Alexandria, Washington. Its purpose is to reintegrate veterans into their communities by means of art and education (Armed Services Arts Partnership, n.d.). The role of veterans in modern society is underlined by demonstrating respect to people who survived traumas, losses, and lifestyle changes. It is the responsibility of every citizen to protect the nation and participate in military affairs in case there is an opportunity.
With time, it is a responsibility of society to protect its veterans and offer them appropriate conditions to improve their lives and knowledge. The ASAP program includes four classes for the integration of different co-curricular activities. Veterans and their families are free to choose if they want to participate in comedy classes in order to encourage their creative thinking and remove any sign of shyness or Improv classes, the goal of which is to promote team-building and cooperation (Armed Services Arts Partnership, n.d.). In addition, veterans may visit storytelling and writing classes where they learn how to share their thoughts, tell about the gained experience, and inform about the peculiarities of US military services.
Program Theory
A logic model is the program theory chosen for the evaluation of ASAP activities. It is frequently used because of its simplicity and accuracy in identifying goals, effects, and other components (Chen, 2015). There are six elements of a logic program according to which personal judgments and observations must be developed (Langbein, 2012). The first step is to identify a problem statement or needs assessment (ASAP helps cover the lack of opportunities for veterans to develop their skills and improve their knowledge).
Then, the identification of program objectives is required (ASAP aims to reintegrate veterans into society and improve the quality of life). The evaluation of inputs or resources cannot be ignored, and ASAP includes the work of volunteers, free donations, and cooperation with different foundations. The activities that are developed by ASAP, create a supportive environment, and develop writing, speaking, and thinking tasks have to be discussed.
The next stage of evaluation is the analysis of outputs, which are the courses, public performances, and different classes. Finally, the outcomes of the ASAP program should be recognized. Veterans can introduce their opinions and share attitudes using their knowledge and skills.
The already developed diagram can be applied as a part of program theory:
Evaluation Goals and Questions
This program evaluation is a combination of systematic studies, the goal of which is to identify the strengths and shortages of program performance, and the chosen management practices. It is expected to identify the objectives of the program and combine them with the reached outcomes and observed results. Sometimes, the effects can be justified, and all planned improvements have certain benefits on its participants.
In some cases, program evaluation shows that developers do not use their potential to reach the goals or demonstrate the high efficiency of the program. Therefore additional recommendations may be given to make sure that all goals are achieved and all questions are answered. In this project, the task is to evaluate the ASAP program with its potential benefits to veterans. The goals of the evaluation are directly derived from the goals of the program.
For example, one of the main program goals is to promote reintegration of veterans into local communities. Regarding this expectations, an evaluation goal is to measure change in veteran social performance before and after education and practices that are offered to them by ASAP.
The second goal of the program is to improve student veterans performance, including their literacy level, critical thinking, and communication skills. In this case, the evaluation goal is to analyze the proficiency level of veterans by ASAP developers regarding their observations and cooperation with veterans at different stages of their education. A final goal of the program is to continue developing new training courses and co-curricular activities in order to increase veterans quality of life and attitudes towards their communities. The evaluation goal is to examine if new activities, social media sources, and stakeholders contribute to the personal and professional growth of veterans.
In total, the main evaluation goal is to check how the ASAP program can change veterans literacy and attitudes towards their social activities and community reintegration by offering new co-curricular activities and involving new people into ASAP courses. Regarding such a wide evaluation goal, there are three major questions that have to be answered in this project:
-
Will the program be able to change veterans attitudes towards life?
-
Will co-curricular activities improve the level of performance among veterans?
-
Does the presence of new stakeholders in the ASAP program promote increased literacy among veterans and improve their satisfaction with the possibility to be reintegrated into communities, using their knowledge and gained skills?
-
Can new training activities provide veterans from the ASAP program with additional opportunities to expand their skills and learn how to develop public speaking, writing, and thinking techniques in comparison to veterans who do not participate in the chosen program?
-
Does the application of credible social media sources create a good chance for veterans to learn more, share their experiences, communicate at a high level, and stay satisfied with living and working conditions after military services?
-
-
Will the level of veteran satisfaction with the offered services increase after improvements?
Data Collection Plan
Any data collection plan has to follow a properly developed research design, indicators, and instruments. The peculiar feature of this program evaluation is the necessity to cooperate with different people and observe the results before and after people join the ASAP program. Therefore, a decision to follow a quasi-experimental design is made for this project. Its essence lies in the necessity to compare the achievements of several groups of people who participate and do not participate in the intervention and identify the worth of the offered idea.
This type of data collection is preferable by many researchers and writers due to its simplicity, validity, and the absence of random sampling (Langbein, 2012). Inclusion criteria have to be established to reduce an endogeneity bias and invite participants who can demonstrate the necessary outcomes and give clear answers to research questions.
A quasi experiment is based on several indicators that measure the effects of the intervention. In this evaluation, dependent and independent variables have to be chosen to investigate veterans activities and attitudes towards education and available opportunities. Training, social media, and stakeholders are the independent variables the essence of which is to use as they are and provoke some changes and formulate new opinions.
Independent variables such as satisfaction, life attitudes, and literacy have to be developed with time after the intervention is offered. In addition to the indicators being given, a quasi-experimental design is characterized by the presence of pretest and posttest activities in order to gather data for program evaluation. These tests are used to identify the opinions of participants and compare their results in regard to a given opportunity (Chen, 2015). As soon as all these concepts and tasks are identified, a plan of data collection can be developed for the ASAP program evaluation and the analysis of the worth of new change mechanisms.
This program evaluation should begin with gathering information about ASAP and the life of veterans in the chosen district. It is necessary to surf the official site of the organization and learn the goals and methods of learning that veterans can use. This source also helps to find out the intentions of its developers, future plans, and the list of organizations that donate, support, and contribute to veterans welfare within the frames of ASAP. As soon as this information is obtained, it is expected to communicate with veterans who can become potential participants in the quasi experiment.
Clear questions have to be developed in written (printed) and online formats. A survey is the main method of gathering data about the program. The survey turns out to be a reliable source of collecting information for evaluating a program within the frames of a quasi-experimental design (Langbein, 2012). It is possible to invite a different number of people, from five people to more than one hundred participants.
Surveys can be organized in three ways: face-to-face (when a researcher poses questions during a real-life meeting and is able to observe reactions and interprets behaviors), telephone (when a researcher uses phone calls and asks participants to answer the questions in real time), and mail (when a researcher sends survey questions via e-mails and communicate with participants online). Regarding such facts as busyness, unwillingness to communicate at certain periods of time, or other personal reasons, some participants cannot answer survey questions in real time.
Therefore, compared to telephone or face-to-face surveys, the mail survey is defined as the most credible and less expensive method of gathering information (Langbein, 2012). It will be used in communication with both veterans and ASAP developers.
However, it is also important to admit that data collection will be developed in three stages. The expected number of veterans to participate in this evaluation is one hundred. They will be divided into two groups experimental and control. An experimental group should join the ASAP program, and a control group remains unknown about this intervention. To avoid misunderstandings, conflicts, and confusion, an experimental group is provided with brief information about ASAP, its purposes, and changes that can be observed in behaviors, emotions, and feelings.
A collection protocol is a significant part of a data collection plan as it includes informed consent with important descriptions of tasks, expectations, and standards. All veterans are informed about the possibility to withdraw the experiment any time they want as free will and autonomy are appreciated. The participants of the experimental group are provided with anonymity and confidentiality so that no personal information is revealed during the evaluation.
The first stage is called pre-implementation and should last for one month. An evaluator focuses on direct communication with veterans who served in American armed forces and are no longer on active duty at the period of communication. It is important to include two groups of veterans: those who participate in the ASAP program and those who have not heard about it. Communication will help to compare the level of veterans knowledge, their literacy, and abilities to express their thoughts. A survey also helps to recognize the current needs and expectations of veterans from their communities.
Sometimes, veterans are not eager to talk about their demands because of the already survived activities, changing morals, and new perspectives. The aim of the pre-test communication is to create a basis for the intervention and describe the main aspects of veteran life. The questions for the pretest stage are given in Appendix 1. Veterans will be able to evaluate their skills and readiness to participate in various activities. They can also observe what they can do or want to do in the future and take a step to make their dreams come true.
The second stage includes the description of the intervention and communication with the ASAP staff. Three months is an approximate period of time that can be spent to gather the necessary material. This data collection cannot be ignored as it helps to answer one the research questions about the level of literacy among veterans and explain the essence of the intervention. The employees of the chosen organization share their opinions about veterans who have just joined the courses and who are going to finish their education.
They compare the level of literacy, the desire to communicate, and the intentions to show personal skills in different fields of art. They can disclose current co-curricular activities and the role of different stakeholders in veterans development. Information about an overall development of the program with its benefits, costs, and possible improvements can be obtained from this kind of communication. Their answers to the list of statements (see Appendix 2) create a solid contribution to the program evaluation. In addition, this stage in the overall data collection plan aims at defining objective opinions and facts about the program, veterans attitudes, and learning abilities.
The third stage of data collection includes a post-test survey for veterans. It will last about one month to make sure that all veterans who participate in the first pre-test survey can join the last stage of the evaluation. A list of close-ended statements will be offered to participants.
The peculiar feature of the survey at this phase is the presence of the statements used in the first stage. Such an approach helps to compare the attitudes of the same people towards the same issues at different periods of time, underlying the worth of the intervention. At the same time, this survey indicates change in the level of literacy and academic performance after the courses of the program are taken. The list of questions that can be posed to veterans at this stage of a data collection plan is properly identified in Appendix 3.
In total, a data collection plan consists of three main tasks: gathering information about the ASAP program, communication with veterans as the direct participants of the intervention, and cooperation with ASAP developers as the main contributors to the improvement of veterans literacy levels. A brief plan of data collection is given in the diagram below:
Data Analysis Plan
The choice of a data analysis plan design is one of the integral parts of any program evaluation. It is not enough to gather credible information; a researcher has to think about how to present this data in a clear and readable way in the project. Therefore, the analysis has to be developed for both qualitative and quantitative information of a quasi experiment. There is a number of specific basics with the help of which all gathered information can be introduced.
Any data analysis plan should include the following steps: the review of evaluation goals, the division of data into qualitative and quantitative with their worth to the project, the analysis of quantitative data with simple counts being used, and the evaluation of qualitative data through content analysis. In the end, the interpretation of the obtained results can be developed to prove the role of the chosen public, non-profit program for veterans.
In the case of this program evaluation, surveys with veterans and the ASAP stuff are the main sources of information. Regarding the fact that all responses are anonymous with a simple language being used, it is easy to analyze the results and obtain a large amount of information in a short period of time. According to Langbein (2012), survey data remains frequently underanalyzed without a possibility of controlled comparison. Therefore, each step has to be properly taken with all the purposes and outcomes being identified before and after the actual process.
First, it is necessary to enumerate all evaluation goals and questions and choose a focus for data analysis. A clear reason why the evaluation is necessary has to be given. In this case, it is the necessity to understand how co-curricular activities chosen for the ASAP courses can change current veterans attitudes towards life, motivate them to develop new skills, and use their experience to express their thoughts and ideas.
This step will help to organize data and use the most appropriate data analysis tools. It is expected to organize data in a chronological order where participants (veterans) share their military experience and life after military services and experience new emotions within the frames of a new program. Then, an outcome-based evaluation must be developed in terms of which all the indicators are mentioned and explained.
The next step of the program evaluation is the identification of data types. To strengthen this evaluation, the researcher must gather enough information about the program from local resources. Census data and vital statics can be used to identify how many veterans are in the country and in the region, and what their major needs and concerns are. The benefit of using these sources is the possibility to work with them online anytime. These records are objective and effective in developing qualitative and quantitative data analysis and interpretation.
The data analysis should include quantitative information about the participants and given by the participants. Quantitative analysis methods are useful as they help to test models and apply simple calculations to answer research questions (Chen, 2015).
Today, researchers can rely on a variety of data analysis software programs that can be purchased, downloaded for free, or borrowed. The most frequently used options are Statistical Package for the Social Sciences, also known as SPSS, and Statistical Analysis System, also known as SAS (Langbein, 2012). In some cases, students prefer to use Microsoft Excel and complete the required calculations.
Quantitative information has to be analyzed in a certain order. First, all questionnaires and answers of the participants should have copies to make sure they can be used in case the original versions are missing or damaged because of some reason. Then, it is important to choose a format to make notes, and many researchers choose to tabulate the information, add numbers, and create ratings. There are one hundred participants in the program evaluation who are provided with anonymity and confidentiality rights. Therefore, it is not expected to explain the answer of each participant but give numeric information like 45 people out of 100 strongly agree with statement 1 or 20 participants out of 50 (control group) disagree with statement 5 .
In this quasi-experimental design, the decision to use a difference-in-differences strategy is made. A basic regression model will help to analyze quantitative data gathered from surveys (Langbein, 2012). The plan is to calculate the frequency of distributions and identify the relationship between an outcome (i.e., veteran satisfaction and literacy level) and the intervention (the promotion of co-curricular activities through inviting new stakeholders, using social media, and promoting training).
Only one group of veterans receives the possibility to participate in the ASAP program, and a comparison group has to be observed during the same period of time without receiving the same intervention. There are four main conditions that have to be identified in the analysis:
-
Ge1 (pre-test condition of an experimental group),
-
Gc1 (pre-test condition of a control group),
-
Ge2 (post-test condition of an experimental group), and
-
Gc2 (post-test condition of a control group). The following calculations have to be made:
-
Ge2 Ge1 = Ge
-
Gc1 Gc1 = Gc
-
Gc Ge = G.
-
In this case, Ge is the difference between what happens to the experimental group before and after the intervention, and Gc is the difference between what happens to the control group before and after the intervention. Finally, G is the difference between the two already-defined differences. The same scheme has to be applied to find the answers to the three research questions posed at the beginning of the program evaluation.
In the end, different results prove the correctness of the chosen strategy and define the worth of co-curricular activities like training, new stakeholders participation, and social media sources from the point of view of veteran satisfaction and literacy level (two last evaluation questions).
In order to answer the first evaluation question about the attitudes of veterans to life, qualitative data has to be gathered and analyzed. In the list of close-ended statements (Appendix 1 and Appendix 3), content analysis can be used to underline the main changes and feelings of veterans before, during, and after the intervention. First, it is important to read all the data gathered during the evaluation and make notes if they are appropriate.
Then, it is expected to underline the main topics and label several categories or themes for future analysis. As soon as the main patterns are identified, the researcher can leave commentary using past studies, census data, or other arguments he/she finds reliable.
Data analysis usually lasts between one to three months after the intervention is complete. There is no possibility to add new information about participants or develop additional answers. No fake answers and participants should be present in the program evaluation. If a smaller number of participants agree to participate, or some people do not meet all the inclusion criteria, the researcher should explain the outcomes and inform the reader about the unexpected changes in the project. As it is a plan of data analysis, the expected number of replies remains 100, including 50 veterans from an experimental group who participate in the intervention and 50 veterans from a control group who do not participate in the intervention.
The final stage of data analysis is to interpret and represent information after it is gathered and analyzed. For example, the comparison of the results has to be combined with the original program goals. The opinions of veterans must be properly developed, the contributions of the ASAP staff have to be mentioned, and the strengths/weakness of the program should be identified. The next step is giving recommendations to promote the improvement of the program and the development of new co-curricular activities with the help of which the level of literacy and satisfaction among veterans can be increased. Finally, conclusions about the work done and the contributions made are given in a clear and justified way. It is important to make sure that all evaluation questions and goals are answered and achieved.
Plan for Working and Communicating with Stakeholders
Program evaluation consists of a number of important steps with their goals, specific methods, and outcomes. To develop strong and effective evaluation, it is important to identify and engage stakeholders and choose appropriate ways for working and communicating. In this part of a plan, special attention will be paid to stakeholders and their role in the program. On the one hand, the work with stakeholders can increase the evaluability of the program for assessment, and, on the other hand, this type of cooperation promotes several pragmatic recommendations to improve the program and develop a responsive evaluation (Chen, 2015). The establishment of working relationships between stakeholders and researchers or evaluators is an integral step in program evaluation.
To be working, stakeholder-evaluator relationships have to be culturally relevant and well-determined. Regardless of the type of program for evaluation, stakeholders are usually defined as people or organizations invested in a program in a variety of ways (Centers for Disease Control and Prevention, 2014). For example, they may be involved with program operations and services and be a part of the program staff or funding agencies.
In this case, the main representatives of this group of stakeholders are the employees of ASAP, as well its funding partner who offer their financial support to the programs development (Barclays, NEX Giving Day, or Advisors Real Estate). Stakeholders can also be the individuals who use the services within the program or are affected by its results. As a rule, they are patients, clients, students, and other community members who are aware of their rights and use their social wisdom (Centers for Disease Control and Prevention, 2014). In the ASAP program evaluation, the opinions of veterans, their families, and local community members as the current or potential stakeholders may define the assessment.
The results of evaluation depend on how well researchers define and understand the perspectives of stakeholders and incorporate them with overall evaluation purposes. However, one may think that the less a stakeholder knows about goals and intentions of evaluators, the better and more objective his/her participation can be. Chen (2015) challenges this approach and underlines that stakeholders participation can shape the scope of the evaluation.
Stakeholders use their knowledge and experience with ASAP in order to formulate effective questions, discuss the chosen evaluation designs, and reflect the reality before, while, and after the evaluation. Their participation turns out to be more effective when people are involved in the process, unless many of them can ignore or criticize the results of evaluation just because their opinions are neglected. Therefore, the rule number one in working with stakehol
Order from us for quality, customized work in due time of your choice.