Relative clauses in English
In English, relative clauses are formed by removing something, leaving behind a gap.
1a) I kicked the ball
1b) I wrote with the pen
1c) I entered the building
1d) I played in the garden
In each example, we'll pull out the bolded noun phrase, leaving behind a gap:
2a) the ball [ which/that/∅ I kicked ____ ]
2b) the pen [ which/that/∅ I wrote with ____ ]
2c) the building [ which/that/∅ I entered ____ ]
2d) the garden [ which/that/∅ I played in ____ ]
Note: the ∅ symbol indicates the possibility of using neither which nor that
In examples a and c, the gaps are the objects of the verbs kicked and entered. In examples b and d, the gaps are the objects of the prepositions with and in.
The gap in each case is indicated by a relative pronoun (who, what, which, how, where, when, etc.), which is inserted at the beginning of the relative clause. In some situations, this pronoun can be replaced with the less specific that, and in some situations that can be deleted entirely.
As it happens, in all four of your examples the appropriate relative pronoun was which; in all four it can be replaced with that; and in all four that can be deleted entirely.
In two examples (b and d) the preposition before the gap can be optionally pied piped—moved before the relative pronoun which—but if you do this, you can't replace it with that:
3b) the pen [ with which I wrote
with ____ ]
3d) the garden [ in which I played
in ____ ]
Because in many cases that can be deleted, you can construct examples in English without any overt indicator of where the gap is:
4a) the ball [ ∅ I kicked ____ ]
4b) the pen [ ∅ I wrote with ___ ]
4c) the building [ ∅ I entered ____ ]
4d) the garden [ ∅ I played in ____ ]
Although there are no words "to link the clause to the main sentence" in examples 4a through 4d, I'm sure you have no problem understanding these examples. You immediately understand where the gap is, even if you don't consciously think about something called a "gap"—or indeed about grammar at all!
Relative clauses in Japanese
In Japanese, not every relative clause has a gap, but most do.
However, there are no relative pronouns like who, what, where, etc.; there are no words like that; and there is no pied piping. This makes gapped relative clauses in Japanese fairly simple. They're very similar to the English examples 4a through 4d above. Since you can understand those, I'm sure you can understand relative clauses in Japanese, too!
Let's take a look at your examples. But before we can relativize them, let's replace the topic particle は with the case particle が. Why? Because relative clauses in Japanese don't contain topics, so you need to use the version with が to turn them into relative clauses.
Okay, here are your non-relative examples with は replaced:
5ａ) 私が ボールを 蹴った
5ｂ) 私が ペンで 書いた
5ｃ) 私が ビルに 入った
5ｄ) 私が 庭で 遊んだ
In each example, we have two case-marked nouns. (That means they're marked with case particles like が, を, で, に, and so on.) Just like in English, we can pick one of these to remove, leaving behind a gap:
6ａ) [ ＿＿ ボールを 蹴った ] 私
6ｂ) [ ＿＿ ペンで 書いた ] 私
6ｃ) [ ＿＿ ビルに 入った ] 私
6ｄ) [ ＿＿ 庭で 遊んだ ] 私
Now we have a relative clause! The head noun in each example is 私, which has been pulled out of the relativized clause. Because in each example the case particle が has gone missing, nothing tells us what role is played by the head noun 私—we need to figure it out from context.
We can tell, for example, that in 6a the relative clause already has an を-marked noun, so the role played by 私 cannot be an を role. We must guess what the role is from the set of possible roles the relative clause does not contain, and then use our common sense to decide.
Of course, we could relativize the other element, as in your examples:
7ａ) [ 私が ＿＿＿＿ 蹴った ] ボール
7ｂ) [ 私が ＿＿＿ 書いた ] ペン
7ｃ) [ 私が ＿＿＿ 入った ] ビル
7ｄ) [ 私が ＿＿ 遊んだ ] 庭
Take a look back at sentences 4a through 4d again. You can understand those, right? They're just like 7a through 7d—there aren't any words specifying the role the head noun plays in the relative clause! And if you can understand those sentences in English, I'm sure you can understand their equivalents in Japanese. In each case, you can figure out which case role makes sense: を, で, に, and で.
There's another process we need to talk about to describe relative clauses in Japanese, by the way. It's called が／の conversion! When you have a noun phrase marked with が in a relative clause, you can optionally mark it with の instead. Let's rewrite the examples from 7a through 7d using の:
8ａ) [ 私の ＿＿＿＿ 蹴った ] ボール
8ｂ) [ 私の ＿＿＿ 書いた ] ペン
8ｃ) [ 私の ＿＿＿ 入った ] ビル
8ｄ) [ 私の ＿＿ 遊んだ ] 庭
But if the relative clause contains an を-marked noun phrase, you can't replace が with の. Example 9a is fine, but example 9b is ungrammatical with the intended meaning:
9ａ) [ ＿＿ ジョンが 本を 買った ] 店
9ｂ) [ ＿＿ *ジョンの 本を 買った ] 店
Overall, forming these case-gapped relative clauses in Japanese is relatively simple. Although their interpretation is sometimes ambiguous, you can usually figure it out from context. In particular, looking at the set of case roles already present in the relative clause can help you narrow down where the gap is.
Gapless relative clauses
However, Japanese also contains relative clauses without a gap. In these gapless relatives, the head noun is in some way semantically restricted by the content of the relative clause. You'll commonly see these with formal nouns like こと:
10ａ) [ 東京に行った ] ことがない
In this example, the relative clause 東京に行った gives the formal noun こと a specific meaning. But obviously, there's nowhere in the relative clause to insert こと; it has no gap!
You'll also see this type of relative clause with lexical nouns. I'll borrow a few examples from Timothy Baldwin's thesis, along with his English translations:
11ａ) [ 勝つ ] 意志 the will to win
11ｂ) [ 魚を焼く ] 煙 smoke from grilling fish
11ｃ) [ 学校に行った ] 帰り on the way back from school
In each example, the precise semantic relationship between the relative clause and the head noun is slightly different, but it can be inferred from context. But in each example, there's no gap to be filled!
So to understand a relative clause in Japanese, ask yourself these four questions:
- Is there a gap? Most of the time there is, but there isn't always!
- If so, which case roles make sense with the verb? (It can't be を with 寝る, for example.)
- Of those, which roles are already filled? (Where can the gap not be?)
- Of the remaining possibilities, which role makes the most sense in context?
Relative clauses are very common in Japanese, and the more you read, the more you'll probably do all four steps automatically. Sure, sometimes things will be ambiguous and you'll have to think about it, but most of the time you'll probably be able to figure it out from context.
My advice? Just get used to spotting the possible relationships, and you'll be well on your way to understanding relative clauses in Japanese!