What can we learn about the early Church from extra-Scriptural sources? To all who think the early Christians got together to sing "Rock of Ages", hold hands, have a piece of mere bread once a year "in memory of Christ," and just generally indulge in a feel-good, non-hierarchical, totally egalitarian "fellowship" with each other, here are a few writings to ponder. Do these words of St. Cyprian of Carthage (baptized ca. A.D. 246) this sound like a non-hierarchical, get-together-to-fellowship kind of Church?:

You have written also, that on my account the Church has now a portion of herself in a state of dispersion, although the whole people of the Church are collected, and united, and joined to itself in an undivided concord: they alone have remained without, who even, if they had been within, would have had to be cast out. Nor does the Lord, the protector of His people, and their guardian, suffer the wheat to be snatched from His floor; but the chaff alone can be separated from the Church, since also the apostle says, "For what if some of them have departed from the faith? shall their unbelief make the faith of God of none effect? God forbid; for God is true, but every man a liar." And the Lord also in the Gospel, when disciples forsook Him as He spoke, turning to the twelve, said, "Will ye also go away?" then Peter answered Him, "Lord, to whom shall we go? Thou hast the word of eternal life; and we believe, and are sure, that Thou art the Son of the living God." Peter speaks there, on whom the Church was to be built, teaching and showing in the name of the Church, that although a rebellious and arrogant multitude of those who will not hear and obey may depart, yet the Church does not depart from Christ; and they are the Church who are a people united to the priest, and the flock which adheres to its pastor. Whence you ought to know that the bishop is in the Church, and the Church in the bishop; and if any one be not with the bishop, that he is not in the Church, and that those flatter themselves in vain who creep in, not having peace with God's priests, and think that they communicate secretly with some; while the Church, which is Catholic and one, is not cut nor divided, but is indeed connected and bound together by the cement of priests who cohere with one another.

In the 1st century, St. Ignatius, Peter's appointee to the Antiochian bishopric, addressed his letter to the Roman Church like this:

...to the Church which has obtained mercy, through the majesty of the Most High Father, and Jesus Christ, His only-begotten Son; the Church which is beloved and enlightened by the will of Him that willeth all things which are according to the love of Jesus Christ our God, which also presides in the place of the report of the Romans, worthy of God, worthy of honour, worthy of the highest happiness, worthy of praise, worthy of obtaining her every desire, worthy of being deemed holy, and which presides over love, is named from Christ, and from the Father, which I also salute in the name of Jesus Christ, the Son of the Father: to those who are united, both according to the flesh and spirit, to every one of His commandments; who are filled inseparably with the grace of God, and are purified from every strange taint, I wish abundance of happiness unblameably, in Jesus Christ our God.

In his letter to the Smyraean Church he wrote:

Let no man do anything connected with the Church without the bishop. Let that be deemed a proper Eucharist [the Catholic word for "Communion"], which is administered either by the bishop, or by one to whom he has entrusted it. Wherever the bishop shall appear, there let the multitude of the people also be; even as, wherever Jesus Christ is, there is the Catholic Church. It is not lawful without the bishop either to baptize or to celebrate a love-feast; but whatsoever he shall approve of, that is also pleasing to God, so that everything that is done may be secure and valid.

In his letter to the Ephesians:

[Speaking of Bishops] For we ought to receive every one whom the Master of the house sends to be over His household, as we would do Him that sent him. It is manifest, therefore, that we should look upon the bishop even as we would upon the Lord Himself. And indeed Onesimus [see Colossians 4:8-10 and Philemon 1:10] himself greatly commends your good order in God, that ye all live according to the truth, and that no sect has any dwelling-place among you. Nor, indeed, do ye hearken to any one rather than to Jesus Christ speaking in truth.

• the Church was Divinely established as a visible society, the salvation of souls is its end, and those who separate themselves from it cut themselves off from God (Epistle to the Philadelphians)
• the hierarchy of the Church was instituted by Christ (Epistles to the Philadelphians and the Ephesians)
• the threefold character of the hierarchy (Epistle to the Magnesians)
• the order of the episcopacy superior by Divine authority to that of the priesthood (Epistles to the Magnesians, Smyraenians, and the Trallians)
• the importance of unity of the Church (Epistles to the Trallians, Philadelphians, and the Magnesians)

• emphasis on the holiness of the Church (Epistles to the Smyraeans, Ephesians, Magnesians, Trallians, and Romans)

• the catholicity of the Church (Letter to the Smyraeans)

• the infallibility of the Church (Epistles to the Philadelphians and the Ephesians)
• the doctrine of the Eucharist -- i.e., belief in Transsubstantiation or the Real Presence of Christ in Communion (Epistle to the Smyraeans)
• the Incarnation (Epistle to the Ephesians)
• the supernatural virtue of virgnity (Epistle to Polycarp)
• the religious character of matrimony (Epistle to Polycarp)
• the value of united prayer (Epistle to the Ephesians)
• the primacy of the Chair of Peter (Epistle to the Romans, introduction)
• a dencouncing of the (later Protestant) doctrine of private judgement in matters of religion (Epistle to the Philadelphians)¹

St. Irenaeus, Bishop of Lyons, born between A.D. 115 and 125 (or between 130 and 142, the date is unclear though it is certain that he met Bishop Polycarp (d. 155) at Smyrna) wrote in his Adversus Haereses Book III Ch. IV about sorting Truth from heresy:

Since therefore we have such proofs, it is not necessary to seek the truth among others which it is easy to obtain from the Church; since the apostles, like a rich man depositing his money in a bank, lodged in her hands most copiously all things pertaining to the truth: so that every man, whosoever will, can draw from her the water of life. For she is the entrance to life; all others are thieves and robbers. On this account are we bound to avoid them, but to make choice of the thing pertaining to the Church with the utmost diligence, and to lay hold of the tradition of the truth.

Which Church was he talking about? The Church built by Christ on the rock of Peter:

Since, however, it would be very tedious, in such a volume as this, to reckon up the successions of all the Churches, we do put to confusion all those who, in whatever manner, whether by an evil self-pleasing, by vainglory, or by blindness and perverse opinion, assemble in unauthorized meetings; we do this, I say, by indicating that tradition derived from the apostles, of the very great, the very ancient, and universally known Church founded and organized at Rome by the two most glorious apostles, Peter and Paul; as also by pointing out the faith preached to men, which comes down to our time by means of the successions of the bishops. For it is a matter of necessity that every Church should agree with this Church, on account of its preeminent authority, that is, the faithful everywhere, inasmuch as the apostolic tradition has been preserved continuously by those faithful men who exist everywhere" (ibid., Book 3, Ch 2, 2).

Footnote
¹ Information from the Catholic Encyclopedia

Relevant Scripture

Matthew 7:21-23
Not every one that saith unto me, Lord, Lord, shall enter into the kingdom of heaven; but he that doeth the will of my Father which is in heaven. Many will say to me in that day, Lord, Lord, have we not prophesied in thy name? and in thy name have cast out devils? and in thy name done many wonderful works? And then will I profess unto them, I never knew you: depart from me, ye that work iniquity.

Matthew 9:8
But when the multitudes saw it, they marvelled, and glorified God, which had given such power unto men.

Matthew 12:25
And Jesus knew their thoughts, and said unto them, Every kingdom divided against itself is brought to desolation; and every city or house divided against itself shall not stand:

Matthew 16:18
And I say also unto thee, That thou art Peter, and upon this rock I will build my church; and the gates of hell shall not prevail against it.

Matthew 18:17
If he refuses to listen to them, tell it to the church; and if he refuses to listen even to the church, treat him as you would a pagan or a tax collector.

Matthew. 28:19–20
Go ye therefore, and teach all nations, baptizing them in the name of the Father, and of the Son, and of the Holy Ghost: Teaching them to observe all things whatsoever I have commanded you: and, lo, I am with you always, even unto the end of the world. Amen.

John 10:16
And other sheep I have, which are not of this fold: them also I must bring, and they shall hear my voice; and there shall be one fold, and one shepherd.

John 17:20-23
Neither pray I for these alone, but for them also which shall believe on me through their word; That they all may be one; as thou, Father, art in me, and I in thee, that they also may be one in us: that the world may believe that thou hast sent me. And the glory which thou gavest me I have given them; that they may be one, even as we are one: I in them, and thou in me, that they may be made perfect in one; and that the world may know that thou hast sent me, and hast loved them, as thou hast loved me.

John 20:21-23
Then said Jesus to them again, Peace be unto you: as my Father hath sent me, even so send I you. And when he had said this, he breathed on them, and saith unto them, Receive ye the Holy Ghost: Whose soever sins ye remit, they are remitted unto them; and whose soever sins ye retain, they are retained.

Acts 4:32
And the multitude of them that believed were of one heart and of one soul: neither said any of them that ought of the things which he possessed was his own; but they had all things common.

Romans 12:5
So we, being many, are one body in Christ, and every one members one of another.

Romans 16:17
Now I beseech you, brethren, mark them which cause divisions and offences contrary to the doctrine which ye have learned; and avoid them.

1 Corinthians 1:10-13
Now I beseech you, brethren, by the name of our Lord Jesus Christ, that ye all speak the same thing, and that there be no divisions among you; but that ye be perfectly joined together in the same mind and in the same judgment. For it hath been declared unto me of you, my brethren, by them which are of the house of Chloe, that there are contentions among you. Now this I say, that every one of you saith, I am of Paul; and I of Apollos; and I of Cephas; and I of Christ. Is Christ divided? was Paul crucified for you? or were ye baptized in the name of Paul?

1 Corinthians 3:3-4
For ye are yet carnal: for whereas there is among you envying, and strife, and divisions, are ye not carnal, and walk as men? For while one saith, I am of Paul; and another, I am of Apollos; are ye not carnal?

1 Corinthians 4:14-15
I write not these things to shame you, but as my beloved sons I warn you. For though ye have ten thousand instructers in Christ, yet have ye not many fathers: for in Christ Jesus I have begotten you through the gospel. [NIV: I am not writing this to shame you, but to warn you, as my dear children. Even though you have ten thousand guardians in Christ, you do not have many fathers, for in Christ Jesus I became your father through the gospel]

1 Corinthians 10:17
For we being many are one bread, and one body: for we are all partakers of that one bread.

1 Corinthians 11:18-19
For first of all, when ye come together in the church, I hear that there be divisions among you; and I partly believe it. For there must be also heresies among you, that they which are approved may be made manifest among you.

1 Corinthians 12:12-27
For as the body is one, and hath many members, and all the members of that one body, being many, are one body: so also is Christ. For by one Spirit are we all baptized into one body, whether we be Jews or Gentiles, whether we be bond or free; and have been all made to drink into one Spirit. For the body is not one member, but many. If the foot shall say, Because I am not the hand, I am not of the body; is it therefore not of the body? And if the ear shall say, Because I am not the eye, I am not of the body; is it therefore not of the body? If the whole body were an eye, where were the hearing? If the whole were hearing, where were the smelling? But now hath God set the members every one of them in the body, as it hath pleased him. And if they were all one member, where were the body? But now are they many members, yet but one body. And the eye cannot say unto the hand, I have no need of thee: nor again the head to the feet, I have no need of you. Nay, much more those members of the body, which seem to be more feeble, are necessary: And those members of the body, which we think to be less honourable, upon these we bestow more abundant honour; and our uncomely parts have more abundant comeliness. For our comely parts have no need: but God hath tempered the body together, having given more abundant honour to that part which lacked. That there should be no schism in the body; but that the members should have the same care one for another. And whether one member suffer, all the members suffer with it; or one member be honoured, all the members rejoice with it. Now ye are the body of Christ, and members in particular.

1 Corinthians 14:33
For God is not the author of confusion, but of peace, as in all churches of the saints.

2 Corinthians 12:20
For I fear, lest, when I come, I shall not find you such as I would, and that I shall be found unto you such as ye would not: lest there be debates, envyings, wraths, strifes, backbitings, whisperings, swellings, tumults:

Ephesians 2:19-20
Now therefore ye are no more strangers and foreigners, but fellow citizens with the saints, and of the household of God; And are built upon the foundation of the apostles and prophets, Jesus Christ himself being the chief corner stone;

Ephesians 4:3-6
Endeavouring to keep the unity of the Spirit in the bond of peace. There is one body, and one Spirit, even as ye are called in one hope of your calling; One Lord, one faith, one baptism, One God and Father of all, who is above all, and through all, and in you all.

Ephesians 5:25–27
Husbands, love your wives, even as Christ also loved the church, and gave himself for it; That he might sanctify and cleanse it with the washing of water by the word, That he might present it to himself a glorious church, not having spot, or wrinkle, or any such thing; but that it should be holy and without blemish.

Philippians 1:27
Only let your conversation be as it becometh the gospel of Christ: that whether I come and see you, or else be absent, I may hear of your affairs, that ye stand fast in one spirit, with one mind striving together for the faith of the gospel;

Philippians 2:2-3
Fulfil ye my joy, that ye be likeminded, having the same love, being of one accord, of one mind. Let nothing be done through strife or vainglory; but in lowliness of mind let each esteem other better than themselves.

1 Thessalonians 2:11
As ye know how we exhorted and comforted and charged every one of you, as a father doth his children

1 Timothy 6:3-5
If any man teach otherwise, and consent not to wholesome words, even the words of our Lord Jesus Christ, and to the doctrine which is according to godliness; He is proud, knowing nothing, but doting about questions and strifes of words, whereof cometh envy, strife, railings, evil surmisings, Perverse disputings of men of corrupt minds, and destitute of the truth, supposing that gain is godliness: from such withdraw thyself.

Titus 3:9-10
But avoid foolish questions, and genealogies, and contentions, and strivings about the law; for they are unprofitable and vain. A man that is an heretick after the first and second admonition reject;

James 3:16
For where envying and strife is, there is confusion and every evil work.

2 Peter 2:1
But there were false prophets also among the people, even as there shall be false teachers among you, who privily shall bring in damnable heresies, even denying the Lord that bought them, and bring upon themselves swift destruction.

Revelation 5:9–10
And they sung a new song, saying, Thou art worthy to take the book, and to open the seals thereof: for thou wast slain, and hast redeemed us to God by thy blood out of every kindred, and tongue, and people, and nation; And hast made us unto our God kings and priests: and we shall reign on the earth.

Revelation 19:7-8
Let us be glad and rejoice, and give honour to him: for the marriage of the Lamb is come, and his wife hath made herself ready. And to her was granted that she should be arrayed in fine linen, clean and white: for the fine linen is the righteousness of saints.

Further Reading

The Four Marks of the Church
From the Catholic Enyclopedia on the Four Marks of the Church: Unity, Sanctity, Catholicity, and Aposolicity