Categories in Portfolio

I am trying to make the Portfolio section show only specific categories that I have set within Wordpress, but regardless of which categories I select, it always shows my four latest posts from all categories. How do I make it show only the category that I select?