Zest one orange on a grater or micro-plane to get 1 teaspoon zest. Set aside until using. Peel the remaining oranges and separate into wedges. Cut the apples into cubes. Combine the orange wedges and apple cubes and place in the refrigerator until cooking.
Pour the cranberries into a medium-sized sauce pan. Cover with sugar and pour in water.
Bring the water to a boil, reduce heat to a simmer. Stir in the remaining ingredients.
Cook for 10 minutes, until thickened. Slightly mash the fruit before serving. Stir in walnuts. Serve hot.