There are 4 different types of questions in RecipeQA. Each of these question types tests a specific comprehension skill. These question types are:
Dataset of the RecipeQA Challenge is held in JSON format so that parsing all context of a question of for a recipe can be done easily. Keys of the dataset are below:
recipe_id: Name of the recipe.context_modality: Contents of the context.context: Context of steps of the recipe.choice_list: Choice list of the question.answer: Answer of the question.qid: ID of the question.question_modality: Question model that can be either image or title.question_text: Question text describing the reasoning task.question: Question clause or question images.task: Type of the question.Structure of the dataset vary according to question type. Therefore, you can investigate the structure of each question type below
{
"recipe_id": "bacon-pancake-poppers",
"context_modality": [
"body",
"title",
"videos"
],
"split": "val",
"context": [
{
"body": "\n -Bacon (of course)\n\n\tThis batch of pancake batter will make approximately sixteen half slices of bacon.\n\tFor the batter...\n\t\t1 egg\n\t\t1 cup flour\n\t\t3/4 cup milk\n\t\t2 tablespoons vegetable or canola oil\n\t\t2 tablespoons sugar\n\t\t1 tablespoon baking powder\n\t\t1/2 teaspoon salt\nDon't worry about having extra batter, it makes a great funnel cake.\n\n\t\tjelly (optional)\n\tMy mom loves bacon and jelly, so when I make them for her I insert jelly.\nYou also need...\n\n\t\ta deep fryer (or just a sauce pan)\n\t\tcanola (or other oil) for deep frying\n\t\tskewers\n\t\ta cookie sheet",
"id": 1,
"videos": [],
"title": "You Will Need..."
},
{
"body": "Main ingredient, BACON!\nBegin by preheating your oven to 375o\nCut eight slices of bacon (or however many your making) in half.\nNext, roll and stick on the skewer. You don't want them touching because they don't cook evenly if they are touching.\nPut them in the oven for 30-45 minutes, turning them over about every 15 minutes. If the pan is over flowing with grease, take it out and pour off the excess grease.\nWhen the bacon is done remove from skewers.",
"id": 2,
"videos": [],
"title": "The Bacon"
},
{
"body": "\n Beat the egg until it is fluffy.\nAdd other ingredients, and stir until smooth.\n\t\t1 egg\n\t\t1 cup flour\n\t\t\u00a03/4 cup milk\n\t\t\u00a02 tablespoons vegetable or canola oil\n\t\t\u00a02 tablespoons sugar\n\t\t\u00a01 tablespoon baking powder\n\t\t\u00a01/2 teaspoon salt",
"id": 3,
"videos": [],
"title": "The Pancake"
},
{
"body": "This is something generally only add if I'm making these for someone who loves bacon with jelly (like my mom).\nTo add jelly simply remove bacon from skewer and unroll. Then, put a small spoonful of jelly in and roll back up.\nOr, if jelly isn't your thing, you could always experiment to find your own favorite fillings.\n---After thought---\nOn this instructables' comment board, Dartssnake suggested using a pastry bag to fill the bacon so that you don't have to roll and re-roll. This should help the bacon maintain it's round shape.",
"id": 4,
"videos": [],
"title": "Jelly (Optional)"
},
{
"body": "Take the bacon bites, and dip into the pancake batter.\nNext, with oil at about 335o, drop bacon with batter into the oil.\nAllow to cook until golden.\nIf you have extra batter, you can just drizzle it into the oil and allow to cook until golden. Then, remove from the oil, and sprinkle with powdered sugar. You have just created a funnel cake.",
"id": 5,
"videos": [],
"title": "Deep Frying"
},
{
"body": "I enjoy dipping these bacony bites in my favorite maple syrup, but the jelly filled ones I eat plain. These aren't just breakfast food! They taste good any time of day.",
"id": 6,
"videos": [],
"title": "Enjoy!"
}
],
"choice_list": [
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_2_2.jpg",
"bacon-pancake-poppers_3_0.jpg",
"home-brew-heater-controller_1_1.jpg"
],
"answer": 3,
"qid": "3000-4995-0-1-2-3",
"question_modality": [
"images"
],
"question": [
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_2_2.jpg",
"bacon-pancake-poppers_3_0.jpg",
"home-brew-heater-controller_1_1.jpg"
],
"task": "visual_coherence"
"question_text": "Select the incoherent image in the following sequence of images."
}
{
"recipe_id": "baked-chicken-and-butternut-squash-tagine",
"context_modality": [
"body",
"title",
"videos"
],
"split": "val",
"context": [
{
"body": "You will need:\n-1 whole, raw chicken, cut into serving pieces (leg, breast, wing, etc.)\n-1 medium butternut squash, peeled and cubed\n-1 15 oz. can chickpeas, drained and rinsed\n-2 cups chicken broth\n-Juice of 1 large orange\n-1 white onion, chopped\n-2 plum tomatoes, diced\n-1/2 cup dried apricots\n-1/2 cup chopped raw walnuts\n-1 Tbsp. olive oil\n-1 shallot, chopped\n-1 clove garlic, chopped\n-1 cinnamon stick\n-pinch saffron threads\n-2 Tbsp. milk or milk substitute\n-1 tsp. Turmeric\n-1 1/2 tsp. Cardamom\n-1 1/2 tsp. Cumin\n-1/2 tsp. Cayenne pepper\n-salt and black pepper to taste (season and taste as you go)",
"id": 1,
"videos": [],
"title": "Ingredients!"
},
{
"body": "Preheat oven to 375.\nYou're going to start out by making a Sofrito, aka a mixture of tomatoes, onion, and garlic (and shallot, in this case).\nHeat the olive oil in a frying pan, preferably cast iron, and add these 4 ingredients. This would be a good time to add some salt and black pepper \"to taste\". Stir until all of it has touched the pan, then turn the heat down to medium-low so it can cook while you mix the spices.\nSmash up the Turmeric, Cumin, Cayenne Pepper, and Cardamom with a mortar and pestle, or just mix the spices together if you don't have a mortar and pestle. Add the mixed spices and cinnamon stick to the Sofrito and stir to combine. It will turn yellow from the Turmeric. How cheerful!\nNow, get ready to turn it even more yellow: Combine the Saffron threads and milk in the mortar and pestle, grind, and let it sit for a couple minutes. If you don't have one, hmmm...maybe use a spoon and small measuring cup?\nAdd the saffron milk, chicken broth and orange juice to the pan and stir to combine. Don't forget to taste to see if it needs adjustments. This is your last chance to taste before adding the raw meat. Remove pan from heat.",
"id": 2,
"videos": [],
"title": "Make the Base"
},
{
"body": "Put the cubed squash, chicken, and chickpeas into a Tagine dish or baking dish with a lid. Top with dried Apricots.\nSpoon the soup base on top. If there's a little more than can fill the dish that's okay, because you might need it for basting (so don't throw it away!)",
"id": 3,
"videos": [],
"title": "Assemble the Dish"
},
{
"body": "\nPut the lid on the dish and bake for 25 minutes. Remove lid and baste if needed.\nReturn to oven and bake 45 more minutes.\nRemove from oven and let sit with lid on for at least ten minutes.\nTop with chopped walnuts. Enjoy the Tagine! Hopefully you aren't sharing it with too many people, because it's great for leftovers.\n*********PLEASE VOTE FOR ME IN THE SOUPS AND STEWS CONTEST!***********",
"id": 4,
"videos": [],
"title": "Bake"
}
],
"choice_list": [
"baked-chicken-and-butternut-squash-tagine_4_0.jpg",
"foolproof-2-minute-hollandaise_2_6.jpg",
"bacon-gravy_3_0.jpg",
"how-to-make-polvoronphilippine-powderd-milk-candy_0_0.jpg"
],
"answer": 0,
"qid": "2000-14613-0-1-2-3",
"question_modality": [
"images"
],
"question": [
"baked-chicken-and-butternut-squash-tagine_1_0.jpg",
"baked-chicken-and-butternut-squash-tagine_2_0.jpg",
"baked-chicken-and-butternut-squash-tagine_3_1.jpg",
"@placeholder"
],
"task": "visual_cloze"
"question_text": "Choose the best image for the missing blank to correctly complete the recipe."
}
{
"recipe_id": "bacon-pancake-poppers",
"context_modality": [
"body",
"title",
"videos"
],
"split": "val",
"context": [
{
"body": "\n -Bacon (of course)\n\n\tThis batch of pancake batter will make approximately sixteen half slices of bacon.\n\tFor the batter...\n\t\t1 egg\n\t\t1 cup flour\n\t\t3/4 cup milk\n\t\t2 tablespoons vegetable or canola oil\n\t\t2 tablespoons sugar\n\t\t1 tablespoon baking powder\n\t\t1/2 teaspoon salt\nDon't worry about having extra batter, it makes a great funnel cake.\n\n\t\tjelly (optional)\n\tMy mom loves bacon and jelly, so when I make them for her I insert jelly.\nYou also need...\n\n\t\ta deep fryer (or just a sauce pan)\n\t\tcanola (or other oil) for deep frying\n\t\tskewers\n\t\ta cookie sheet",
"id": 1,
"videos": [],
"title": "You Will Need..."
},
{
"body": "Main ingredient, BACON!\nBegin by preheating your oven to 375o\nCut eight slices of bacon (or however many your making) in half.\nNext, roll and stick on the skewer. You don't want them touching because they don't cook evenly if they are touching.\nPut them in the oven for 30-45 minutes, turning them over about every 15 minutes. If the pan is over flowing with grease, take it out and pour off the excess grease.\nWhen the bacon is done remove from skewers.",
"id": 2,
"videos": [],
"title": "The Bacon"
},
{
"body": "\n Beat the egg until it is fluffy.\nAdd other ingredients, and stir until smooth.\n\t\t1 egg\n\t\t1 cup flour\n\t\t\u00a03/4 cup milk\n\t\t\u00a02 tablespoons vegetable or canola oil\n\t\t\u00a02 tablespoons sugar\n\t\t\u00a01 tablespoon baking powder\n\t\t\u00a01/2 teaspoon salt",
"id": 3,
"videos": [],
"title": "The Pancake"
},
{
"body": "This is something generally only add if I'm making these for someone who loves bacon with jelly (like my mom).\nTo add jelly simply remove bacon from skewer and unroll. Then, put a small spoonful of jelly in and roll back up.\nOr, if jelly isn't your thing, you could always experiment to find your own favorite fillings.\n---After thought---\nOn this instructables' comment board, Dartssnake suggested using a pastry bag to fill the bacon so that you don't have to roll and re-roll. This should help the bacon maintain it's round shape.",
"id": 4,
"videos": [],
"title": "Jelly (Optional)"
},
{
"body": "Take the bacon bites, and dip into the pancake batter.\nNext, with oil at about 335o, drop bacon with batter into the oil.\nAllow to cook until golden.\nIf you have extra batter, you can just drizzle it into the oil and allow to cook until golden. Then, remove from the oil, and sprinkle with powdered sugar. You have just created a funnel cake.",
"id": 5,
"videos": [],
"title": "Deep Frying"
},
{
"body": "I enjoy dipping these bacony bites in my favorite maple syrup, but the jelly filled ones I eat plain. These aren't just breakfast food! They taste good any time of day.",
"id": 6,
"videos": [],
"title": "Enjoy!"
}
],
"choice_list": [
[
"bacon-pancake-poppers_5_0.jpg",
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_4_1.jpg",
"bacon-pancake-poppers_6_0.jpg"
],
[
"bacon-pancake-poppers_6_0.jpg",
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_5_0.jpg",
"bacon-pancake-poppers_4_1.jpg"
],
[
"bacon-pancake-poppers_4_1.jpg",
"bacon-pancake-poppers_6_0.jpg",
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_5_0.jpg"
],
[
"bacon-pancake-poppers_1_1.jpg",
"bacon-pancake-poppers_4_1.jpg",
"bacon-pancake-poppers_5_0.jpg",
"bacon-pancake-poppers_6_0.jpg"
]
],
"answer": 3,
"qid": "4000-4995-0-3-4-5",
"question_modality": [
"images"
],
"question": [
"Choose the correct order of images to make a story"
],
"task": "visual_ordering"
"question_text": "What is the correct order of the images?"
}
{
"recipe_id": "dessert-hamburgers",
"context_modality": [
"body",
"images",
"videos"
],
"split": "val",
"context": [
{
"body": "Yellow cake mix\nVanilla Ice cream\nHersheys chocolate Shell\nCool Whip\nCarmel Sauce\nStrawberry glaze (for strawberry shortcake)\nCoconut\nGreen food coloring (I used blue and yellow because my mom didn't have any green)\nCupcake pan\nPan grease\nOptional: Red and yellow ketchup and mustard bottles",
"images": [
"dessert-hamburgers_1_0.jpg"
],
"id": 1,
"videos": []
},
{
"body": "Get out your cupcake pans, grease, and yellow cake mix. Grease pan. Follow instructions on the back of cake mix box. If it says to have wrappers around the cupcakes when baking, don't. After taken out of the oven and cooled, cut in half. Lay aside. (I made them the day before)",
"images": [
"dessert-hamburgers_2_0.jpg",
"dessert-hamburgers_2_1.jpg",
"dessert-hamburgers_2_2.jpg"
],
"id": 2,
"videos": []
},
{
"body": "Have Shell ready for use. Get a cup out for easy cutting (you will have to have a slab of ice cream cut into circles).Get your box of ice cream out with a spreading spatchula. Cut a fairly thin slab of ice cream with the spatchula. Use cup to take out small circles of ice cream. After every two circles, cover one side with Shell. Place each two\u00a0on a\u00a0plate (two per plate). Stick in freezer after evey plate is complete. I spread it with my hand but I had the idea after we were done to put the Shell in a pan and dip the ice cream circles in it. After it's complete, take each plate out, flip the ice cream and do the other side. You will have ice cream patties when finished.",
"images": [
"dessert-hamburgers_3_0.jpg",
"dessert-hamburgers_3_1.jpg",
"dessert-hamburgers_3_2.jpg",
"dessert-hamburgers_3_3.jpg",
"dessert-hamburgers_3_4.jpg"
],
"id": 3,
"videos": []
},
{
"body": "Place half of the coconut in a bag with some green food coloring. Shake bag until green. Put each white and green coconut in a bowl. Place in fridge.",
"images": [
"dessert-hamburgers_4_0.jpg",
"dessert-hamburgers_4_1.jpg"
],
"id": 4,
"videos": []
},
{
"body": "Optional: Place carmel in mustard bottle and glaze in ketchup bottle.",
"images": [],
"id": 5,
"videos": []
},
{
"body": "Have everyone come in and put there own together because some don't like certain things (I put everything on mine).\nEnjoy!",
"images": [
"dessert-hamburgers_6_0.jpg",
"dessert-hamburgers_6_1.jpg"
],
"id": 6,
"videos": []
}
],
"choice_list": [
"Dessert Bagel",
"Method",
"Ingrediants & Supplies",
"Ingredients / Equipment"
],
"answer": 2,
"qid": "1000-12378-0-1-4-5",
"question_modality": [
"title"
],
"question": [
"@placeholder",
"Cupcakes (buns)",
"Carmel (mustard) and Strawberry Glaze (ketchup)",
"Finished Project"
],
"task": "textual_cloze"
"question_text": "Choose the best text for the missing blank to correctly complete the recipe."
}