On Evaluation

(and sorry for being so very, very late with this…)

My kids, aged 14 and 12, often discuss their grades. Actually, in Norssi alakoulu, no numbers are used, instead, the evaluation is given as verbal feedback. In yläkoulu, however, the kids are evaluated numerically – finally, as my kids feel! It seems to be very important to have grades, somehow more adult maybe. Be as it may, the fact is that also the students in alakoulu often transform their verbal evaluation into numbers.

Not only youngsters, but also professional pedagogy people seem to appreciate numeral grading, which never seizes to baffle me. I belong to a group planning and giving a nationwide blended learning course (partly face to face, mostly in Moodle) in feminist pedagogy. In the group, my senior colleague, actually someone who in some sense really brought feminist pedagogy to Finland in 1990s, has recently participated in a university pedagogy course (YPE), a course for the personnel at the University of Helsinki. As a part of the course, there was a group work (test) that was evaluated with numbers (5: excellent; 4; very good etc.). My colleague, with her group, got a 5, and she has been bragging about it ever since. And, when asking, she says that it was not that their work was appreciated, but exactly the number 5 that mad her happy, and it would not have been the same, if the feedback was “only” verbal.

Having myself been a really “good girl” all the way from ala-aste to university and aiming at the best grades always I do see her point. This notwithstanding, after being forced to give grades to university students as an assistant, and later, a lecturer, and especially along the several pedagogy courses I have taken, I have started to feel very uncomfortable with grading. I know how arbitrary numbers really are. I have realized that learning cannot be measured from outside, and that every effort is more or less doomed to fail.

This of course has to do with my epistemological position. I suppose I have internalised the post-modern or post-structuralist – a social-constructivist – idea of knowledge as not something “truthful” and “pure”, but instead as a process, and as co-created. I do not think it is even possible for a teacher to “feed” the students with existing knowledge. As a teacher or a supervisor in university context, I understand my role as a guide or as someone who is taking part in the process of co-creating knowledge. I may at best be able to give advice, to be consulted, about how to create knowledge, but I cannot simply hand any coherent package of knowledge to anybody. It is for them (us) to produce. Knowing is a process.

So, when giving grades to students, what should I evaluate? Especially in an EMI context: what is it that I am supposed to measure? The content, not the language skills, I suppose, but can I separate these two (especially after confessing myself in my first blog that I feel really stupid with my limited English language skills as I cannot properly explain anything a bit more complicated than the basic everyday mundane stuff in English).

After expressing my doubts, I have to admit that grades may be valuable in motivating students. To take another example of my life: perhaps it has been too easy to leave the assignments undone during this TACE course just because I have known from the beginning that I will not be given a grade. This may be true on the one hand, but on the other hand, it has been a relief that the teachers of this course have been flexible and patient with all the delays related to my performance. It is probably because you know and share the experience of not having enough time for all the tasks one is expected to do.

And since the contemporary university institution is based on utilizing the employees and giving almost nothing (such as positions, tenure) in return, even if the course was graded, I would have perhaps thought that a moderate OK number will be enough. The constant hurry and pressure has really swiped away the girl that always aimed at having 10 as her grade. Today, it is simply impossible, if one is about to sleep at night.

The thoughts above were brought about by Atjonen’s (2007) thoughts on ethics in assessment. It is truly important to ask the following questions related to assessment:

  • what is assessed
  • why (is the aim to motivate, give feedback, or correct mistakes)
  • who does the assessment (the teacher, the student herself, peers)
  • how is  the evaluation done
  • when is it done: it might not be pedagogically wise to give the evaluation only after the course when it is too late to change anything; some evaluation during the course, for example in the middle of it, might be in order to emphasise the processual nature of learning.

It is also crucial to pose the question of what could ethical assessment be. Would it take into account the differences in the classroom? The variety of teaching and learning methods, the variety of personalities and their compatibility? How could it be made sure that the evaluation is fair and based on justice instead of biased observation or rewarding the students to be similar with the evaluator (which is called cultural cloning)? Do we actually assess language skills instead of progress? Do we allow sufficient emphasis on different backgrounds? Can we guarantee that different does not equal wrong?

All in all, the basic question that everybody – no matter whether assessing or being assessed – should ask, is: are we doing the assessment of learning OR assessment for learning? If it is the former, how could we change it into the latter?

Use of Privilege Walk as a tool for teaching

‘’Privilege walk,’ another reflection based activity presented by another team in the class.  I liked this task, as we can use such a task in multicultural classrooms to break the ice. It can also help students to understand how diverse different cultures can be, while, the basic human emotions remain the same. I think teachers can customize the questions of this task based on the needs of the subject. It also brings awareness to everyone in the class about various other possibilities about a topic based on an individual’s reaction to each question.

In CEM classes, I can use this task to highlight different socio-economic issues in different cultures to highlight how multinational corporations can customize their corporate social responsibility geographically. Often, students from one geographical region are not entirely aware of culture and socio-economic problems of students from another geographical region. Sometimes in a multicultural classroom setting, a task like this might help students to discover the background of other students and create an environment to discuss and learn about different issues on an international platform so that the students are more aware of various cultural issues and deal with them appropriately in their future jobs.

I also agree to the following class discussion that as a teacher we need to use this task in classrooms with responsibility. There might be a risk of raising some issues, which can hurt or shock some students. I feel that it would be appropriate that if I provide complete context to the students and make them understand that it is essential to be aware of how other cultures function and this awareness might help them to deal with situations in a better way in future without getting overly judgemental about others or feeling shocked.  

Exposing students to potential risks through role-play in classroom

In business school, role-play can serve as a useful simulation tool for preparing students for real-life business situations. In this post, I am sharing an example of a role-play simulation that we used in one of our course. Based on this I will reflect on the preparation that is involved in organizing such a role-play in a classroom setting.

We used role-play in the course “Stakeholders in networks.” Tiina Onkila, Adrienn Károly, and I teach this course together. Broadly speaking, this course is about how stakeholders can influence business and how businesses engage with its stakeholders. In the last session of this course, we have a role-play exercise to expose the students to the real-life stakeholder negotiation process. Some of the learnings of this exercise are how stakeholders influence businesses, what are the potential risks to business and how two very diverse stakeholder groups might be interrelated to each other.

Possible pointer for planning such a role-play session in a class:

1) Choosing a topic for role-play: It is essential that the students are comfortable with the issue or else the learning outcome will not be that effective. During the first session of our course, we asked students to choose one real-life case from a set of a few real-life cases we provided them.

2) Providing information about the issue. In our course, the students had the owners to find information about the problem. We facilitated this process by presenting them with the existing literature from different stakeholder perspectives.

3) Assigning roles to the students. In our class, we assigned students to different stakeholder groups through the lottery. Each stakeholder group has three to four students. Role assignment was done a session before the role-play session. Delaying the role assignment is debatable. I prefer to postpone until a class session before the actual role-play. I feel this allows students to reflect on all the stakeholders during the course and yet get some time to discuss issues of a specific assigned stakeholder group with other students of the same group.

4) Setting some broad rules for the role-play is essential to facilitate is stakeholder discussion/negotiation.  If there are stringent rules, then students lose creative liberty to try something different from what has happened in the real work.

5) Receiving and giving feedback after the session. In our course, last time, we did not have a formal feedback discussion with all the teachers. However, I do believe Adrienn had the opportunity to discuss this experience in one of her class. I feel after the role-play; we should have a discussion, which may highlight even bigger emotions and issues that the students felt during the role-play.

These were some significant practical issues to prepare for having a role-play session in the class.

Teaching Through English Medium

Teaching becomes challenging when you have students from various cultures and countries in an English medium class; Besides, their interest in learning and improving English at the same time compel me to modify learning experience in the class.

Below are some of the practice which is being ensured an analysed to seek proper response for communication curriculum through English.

a)    Explicit Instruction for the course outline and group assignments

b)    Pre-readings (Case Studies and study material) + reading time in class

c)    Group discussions in class

d)    Group presentation at the end of the course

e)    Group Project (written project report) at the end of the course

f)    Class discussions

Explicit Instruction for the course outline and group assignments

It is essential for any English medium course to put forward explicit instruction regarding course. If a teacher does a great job howsoever in summarising the course outline and related events, despite there can be questions regarding course instructions and confusion. Therefore, I discuss each instruction clearly in the class and ask for the confusion if any.

Pre-readings (Case Studies and study material) + reading time in class

It depends on the course requirement, as far as my course is concern, it requires case studies readings, at least one for each lecture which follows the theoretical concept. In the instruction, home reading is usually mentioned, but I give time in class as well so that, students can read and try to understand within the groups (groups are permanent and declared at the start of the first session).   

Group discussions in class

Group discussion in the class is significant when it comes to providing an equal chance to the student who want to or wishes to participate during the discussion

Group presentation at the end of the course

Group presentation and groups are designed in such a way in my class that they less likely get chances to be with the same language mates; the idea is to compel them to speak English while arguing or justifying their inputs.

Group Project (written project report) at the end of the course

The project report is briefed briefly and also being discuss in the class; the concept here is, that every member of the group has to participate in writing project report so that written English also get chance for the improvement.

Class discussions

Class discussion is offered to students most of the time, the major portion of the lecture are kept for the debate such as case studies and random topic and discussion related to that.

Above are some the tools which are meant to support EMI in my courses; at the same time, it also depends on the audience of the course; therefore, the kind of EMI tools vary.

Interculturality in Business Education

Interculturality has now become the essential part in the business education field; the multinational firms and those planning to internationalize are keen to recruit human capital with multicultural understanding. Graduates who have experiences on their resume about interculturality possess an advantage among others.

Business and trade are different globally, so do are the customers and consumers. It is therefore very important for companies to understand the requirements of the market and in the long run to maintain the credibility of the product or services.

Likewise, it has been a regular practice of academic faculty in the business school field to nourish the students in the class teaching with interculturality tactics. In class room teaching it very essential for the students to form the groups with different nationals or with exchanged students. These groups are then provided with simulation tasks through case studies followed by intensive discussion and opinions. Project report are also one kind of class room assignments that help students to research about the certain market. Such activity provide student with much needed information regarding intercultural prospects.

The teacher responsibility is to provide the latest examples of development in the business and trade; and explain in details why does intercultural skills are important and at the same tolerance to culture, faith, race and religion. Therefore, it is highly recommended in to encourage a multi-national group to participate together.

Digital Technology and teaching

The digital technology and innovation have provided opportunity to the education system by providing equal opportunity to all the students through online education. The trend has been rising for offering online education and to some extent made compulsory for teach to broadcast their online classes for the broader audiences.

Online education although involve certain hardware components, this requirement is opportunity as well as dis advantage. Opportunity for those who have access and affordability where in certain part of the world, there are education seekers who might not be able to cope with the financial condition to available such environment and hardware to access online education.

Every innovation has bright and dark side, though we should encourage the brighter side to overcome the dark side. Online education and courses offered are then modified with the class room teaching, the dry traditional way of class room teaching might to impact high per se.

Therefore, online course require certain understanding of providing online classes including some activity based teaching which are essential for student to learn in better way.

On assesments and feedback

Today (3.12.2018), the whole morning was spent on assessment and feedback – tricky topics indeed. The pre-given instruction seemed rather complex at first. To be honest, the two articles we were required to read, both coming from a discipline I am not at all familiar with, were filled with weird concepts and thought patterns that initially seemed almost incomprehensible and, for a second, made me think that they had to be some kind of a demonstration of what academic English can seem from an outsiders perspective.

The articles were about assessing written tasks and on offering feedback. This, and especially the assessment has always been quite hard for me. I am aware of Bloom’s taxonomy and such aids but my own field, political science, often seems much more like a form of art to me. It is not an exact ”science” in any way, and for me, even the academic theories and philosophies of politics are much more about the rhetorical and argumentative quality. The theories and their arguments can be seen as forms of politics in themselves. As a teacher and a pseudo-thinker of politics then, I tend to accept (almost) all kinds of differing views as valid as long as they are argued well.

Again, the group discussion was very fruitful and my group members had developed various innovative and interesting written assessments for their courses. Me, coming from the traditional, amphitheatre style of lecturing of the humanities, felt like an ancient relic with my lecture diary. But, being bombarded with these new methods during the whole TACE has been and eye opening experience. It has made me think again and again how I could develop my own courses.

On assessments we touched on the subject of the grades. I argued against the pass/fail -style of assessments for two reasons. First, I think it is unjust, since the ones who ”bust their asses” and the ones who are there simply for the credits may get the same result. This is especially complicated if a course is heavily based on group work and the lazy members of a group can pass the course with the help of the others. From this it follows that the grades have an important incentivizing effect: they will encourage everybody (whether individually or as a group) for working hard and for maximizing their learning. Taking an analogy from political sciences: The Soviet Union failed exactly because work was not incentivized, and both the free-riders and the hard-workers of the society ended up having the same, lousy benefits.

A member of my group argued that perhaps the feedback from the teacher could work as an incentive for the student. Well, maybe in a really small and intimate setting, but in a mass lecture type of a course, with one hundred attendees, a truly deep and instructive feedback would hardly be possible.

December 3, Analyzing Writing Tasks: Pre-task

Since I could not participate in our last TACE meeting, I will submit the pre-task in blog form. The reason for not participating is a broken ankle and a casted foot, resulting from a running accident. Below, I am analyzing the portfolio instructions for the ongoing course CCSE3022 Popular Culture Texts, which I am teaching at the University of Vaasa in the role of Doctoral Student at the University of Jyväskylä.

  1. Setting and time: Is the assignment to be written in class or at home? How much time do the students have to complete the assignment? Can the deadline be made flexible for students with special learning needs (e.g. extended deadlines)?

Context: Reflective Portfolio instructions for CCSE3022 Popular Culture Texts, University of Vaasa.

Time: Course time Oct 2 – Dec 11, 2018.  Deadline for the portfolio Jan 15, 2018. Time to complete the portfolio: 5 weeks.

The deadline can be shortened and extended.

  1. Nature of work: Is the assignment individual or written in collaboration (in pairs/groups)? Are there guidelines for possible problem scenarios in the case of collaborative work? (What happens if…?)

The assignment is individual, but the students may enter into a dialogue with other students’ comments and posts, the course material, the teacher’s comments, the world outside the university.

  1. Length of product: Is there a minimum and/or maximum length you set?

The length is specified for the introduction (1 paragraph), the critical reflection (1-3 paragraphs) and the conclusion (1 paragraph). The main body consists of the students’ personal blog post and the length may vary slightly from one student to another, depending on how long their posts are and if they have completed extra credit assignments.

  1. Alignment with learning outcomes: How is the assignment aligned with the learning outcomes of the course?

The learning outcomes of the course are that student can read and analyze different popular culture texts and understand their functions and impact on society. The students are familiar with theories and research on popular culture together with understand and use relevant terminology.

The assignment is directly aligned with the outcomes. During the course the students have been blogging about five different popular genres. The portfolio puts their learning experiences (the blog posts) together into one portfolio text, in which the students will critically reflect on their own writing and learning experiences.

  1. Information sources: Do the students have to use sources? Is there a minimum number of sources you require? Are the students given a reading package or are they required to find their own sources? What types of sources do you expect them to use? What citation style is required?

The students are only required to use “internal sources,” found on Moodle. Since both popular culture texts and research articles on the different genres are available on Moodle, no external sources are needed to complete the portfolio, but naturally it is possible to include external sources if a student wishes to do so. Proper academic referencing is required for the introduction and the critical review but since the actual blog posts should remain intact, no referencing is needed inside these posts unless the students used referencing already in their posts. A Works Cited list is inserted as the last page of the portfolio.

  1. Method of assessment (formative or summative): Do you assess only the final product (=summative assessment) or also earlier versions (=formative assessment)?

 The students are graded on both their blog posts, which constitute the formative assessment, and their reflective portfolios (the summative assessment).

  1. Assessment criteria: What are the assessment criteria that you use? How are they aligned with the learning outcomes of the course? Are the same assessment criteria used in your department or do you use your own? When and how were the criteria developed? Are you satisfied with the criteria or would you like to modify them? If so, how?

The criteria are as follows:

  • Pride and Effort
  • Structure and Organization
  • Grammar and Mechanics
  • Content
  • Documentation
  • Theory, terminology, Concepts
  • Evidence of Thinking/Problem Solving
  • Evidence of Reflection/Self-Evaluation
  • Evidence of Progress/Growth

Borrowing from multiple sources, I have developed my own criteria. I developed the criteria specifically for this course. Since they are language students, creating a reflective portfolio on popular culture, and since they are asked to include all blog posts (not only a selection of their posts), I had to modify the assessment criteria from several sources.  I think it will work, but since both the criteria and the course are new, I will continue to modify them if needed. The rubric is aligned with both the objectives of the course and the portfolio.

  1. Type of feedback: How and when do you provide feedback? What kind of feedback do you provide: oral/written; individual/group; analytic (based on detailed descriptors)/holistic? Do you use peer feedback? If so, how? Do you require the students to submit self-evaluation?

The students have been given personal critical feedback on every blog post (the formative assessment) through written blog comments. The comments are given by me. In addition, the students have written peer feedback twice on their student friends’ posts. The reflective portfolio is a self-evaluation of their own work, in which they reflect on what they have learned in relation to the teacher’s and the other students’ feedback.

  1. Transparency of the assessment criteria: Do you show the students the assessment criteria beforehand? Do you give students a separate list of criteria (or a feedback form) that they can use in the peer-feedback?

The students have the objectives of the portfolio. At this point, the students have not seen the assessment criteria, but I may show it to them on our last class meeting. No feedback form was used for the peer-feedback since the questions and the assignments together with the content varied greatly but could be provided in the future.

  1. Main educational purpose and genre: What is the main purpose of the assignment (broad social function of university education)? Look at the table below (Nesi & Gardner, 2018, p. 53). What genre family does the assignment belong to, and what is the specific genre? How is this genre generally understood or defined in your field (e.g. purpose, organizational structure, balance between descriptive and critical writing, expression of own voice)? Is the expected genre expressed explicitly or implicitly in the writing prompt?

The main purpose of the assignment from the perspective of the broad social function of university education is developing knowledge and understanding of popular culture texts and their functions and impact on society together with developing powers of informed and independent reasoning.

Genre family                specific genre:

  • exercise                        individual blog posts                     implicitly expressed
  •  (self-)critique               portfolio                                         implicitly expressed

Due to my accident, I was not able to complete the points 11-13 in the assignment.

On our teaching activity and on technology in general

Today (12.11.2018), we presented our teaching activity, which hopefully demonstrated the myriad possibilities of digital platforms for collaborative academic projects. The activity went quite well and all the groups were able to create collective libraries (in Mendeley) and comment each others’ articles. Planning and executing this activity made me think more generally of the possibilities and limits of similar kinds of platforms for teaching purposes. Many teachers already use message boards and even chat-applications (such as Slack) in their teaching, and it makes me wonder if I too should update my devices for the 21th century..

Yet it is interesting how these platforms can, in no way, compete with actual human to human interaction, for example, small group discussions or other, rather analog technologies. I’ve taken part in many courses which have used message boards but the disussions have always felt rather forced and bored. They’ve never really taken off, so to speak and I’ve never gotten almost anything from them. Using message boards or chats is somewhat easy for the teachers however as posts and comments are easy to quantify…

The same comparison goes to lecturing with fancy, flashy and colourful powerpoints and other multimedias, which are nowadays tought to be an obligatory part of all teaching. Yes, they can help, but at the same time I can clearly remember the legendary lectures of professor Taneli Kukkonen, who simply walked into the classroom with a coffee cup in his hand and started to talk about the history of ideas in the Islamic world without any notes or what so ever. Sometimes he would write something on the chalkboard, maybe some crucial years or names, but that was it. Without a structural aide such as a powerpoint, he also occasionally drifted away from the core topic, but at all times he was sharp, captivating and conversational with his students. His method and his devices were not that much differen, I can imagine, from the good teachers through all ages of history.

I remember him and his wildly running arabic script on the chalkboard but I have no such memories of any powerpoint presentations.

And I am not a luddite by the way.

our teaching activity and that on intersectionality

Having skipped a number of meetings due to my extensive travelling this autumn, I feel the more so responsible staying on top of things/issues/points that have been covered in TACE. I’ll try to account for these in my following post. This one will be a reflection about last Monday’s meeting.

To start with, preparing for our own teaching activity was a great experience. The initial decision about the topic was rather quick. But it was followed with an intensive process of negotiating many points, not limited to term (though you cannot image how much time it took us to select the final ones that you saw on the slides).

More difficult was to build our own joint conceptual understanding of ethical assessment in a culturally diverse classroom (another term that we negotiated) and to demonstrate this complexity in a (hopefully) clear way to the colleagues in TACE. We also planned the activity as a decidedly unethical task (hopefully it was understandable to others, too =) in order to enable discussion of ethics of assessment and reflection of assessment practices from TACE colleagues.

Thank you everybody for your beautiful poems and the lively discussion!

Highly recommended read for those interested (in Finnish): Atjonen, P. (2007). Hyvä, paha arviointi [ Good, bad assessment]. Helsinki: Tammi.

Happy to hear any further feedback on the lecture and the practical activity, too.

I also enjoyed thoroughly the lecture on intersectionality, which was about a different complexity, and how various factors, all intertwined, create individual experiences, with all their privileges and vulnerabilities, building then a strong argument for considering them together.

I found the activity very fitting for illustrating what intersectionality is and its importance as a conceptual understanding of social justice, fairness, and, closer to our specific context, culturally diverse classroom.

Being a practical person, the first thing I thought about was how to adapt it for my context, and I found it rather easy to (perhaps, depending on who the learners are and how vulnerable they feel about some of their experiences, with modifications as to whether others could see people taking steps forward or back).