Yes, body butter is generally thick.
According to our reference, body butter is described as "a thick, creamy substance that you smooth on to your skin to help keep it supple, moisturized, and looking great." This clearly indicates that thickness is a defining characteristic of body butter.