Boil the potatoes in salted water until fork tender. Drain and transfer to a large mixing bowl.
In a small mixing bowl (you can do this while the potatoes are cooking), mix yogurt, buttermilk, Dijon mustard, rosemary, nutmeg, salt and pepper.
Add that mixture to the cooked potatoes and mix with a potato masher until well incorporated and the potatoes are sort of half mashed (you need to have big chunks of potatoes remaining)
Gently stir in goat cheese, walnuts, green onion and parsley.
Serve warm. (Although this this salad is also excellent cold)